Keywords

1 Introduction

“We’re in 100% digital film space now. I think the industry has to accept that this is like the transition to talkies—it’s massive and it’s game-changing and it’s happening”, insists the famous production designer and director of the USC World Building Media Lab (WBML), Alex McDowell, and continues further to elaborate the new role of digital technology in the filmmaking by claiming: “I expect it can do anything I imagine” [1]. While I believe this argument describes the excitement within the film industry on the doorstep of digital technologies, the possibility to create anything imaginable seems plausible considering the strength of the current visual effects (VFX) industry. However, a question remains, what “100% digital film space” might mean? In McDowell’s own words, we can define this concept as “a non-linear workflow within an immersive technology-driven space” [2].

The development of digital cinema has been noticeably fast. I am writing this paper driven by my own artistic experiences within the field of film production design. I started my career in the 80s by hauling the huge glass sheets with matte paintings on the roof of an old factory building since the visual effects (VFX) were finalized on the actual filming site. I gradually proceeded through the years to use the computer-generated imagery (CGI) in my designs for the first time and was happy enough to add simple flowers on the grass field in a single film scene. However, the possibility to retouch a static image soon developed towards more sophisticated use of camera tracking systems allowing the CGI in the moving shot. Even though the budgets in my country do not allow the filmmakers to use similar comprehensive applications as in blockbuster movies, I eventually could do my filmmaking experiments with extensive use of virtual applications.

Having witnessed this evolution from a personal viewpoint of a practitioner has led me to delve deeper into digitalization. The rapidly changing arena of CGI has undoubtedly provided an escalating amount of techniques to produce and manipulate film image assets, but what about cinema as a whole, as an artform mimicking life itself? As McDowell’s words hint, filmmaking has been transferred from an arduous world of physical limitations into the universe, where only bits matter. Most likely, this crucial change has profound level effects not only on the practice itself, but also on how we establish our presence in the cinematic story worlds, that are essentially synthetic, and thus isolated from the physical experience.

In the following article, I wish to reflect on this transition, where new technologies and artistic expression are intertwined. First, I will be discussing digital cinema from contrasting perspectives. Each of them provides some cue, how CGI related film environments can be understood. However, traditional film theories (such as realist or structuralist film theories) or even the early views on digital cinema seem not to provide sufficient explanation, what does it mean to be in 100% digital film space – practice wise and artistically. Therefore, I proceed to contemplate whether the relatively new approach, enactive cognitive science could provide a novel way to scrutinize digital cinema. As an example, I will introduce the idea of virtual production.

Digital cinema in my mind tends to problematize such concepts as reality, illusion (as false perception), and simulation (as an imitation of reality), so they will be in focus in this essay. The characterization of cinema as moving-image art will also be emphasized since through movement, the cinematic worlds are explored and substantiated.

2 Camera Viewing Space

The cinema as an eye, or a kino-eye, a man seeing through the camera the world, implicating the essence of real, is well expressed through Jean-Luc Godard’s famous analogy suggesting cinema as “truth 24 frames per second” [3]. This notion of film recording the real and experienced is a central argument of the realist film theory, and forcefully put forward by one of its central thinkers, André Bazin. He saw the ontology of a photographic image as a derivative of a technological process incapable of deceiving:

“This production by automatic means has radically affected our psychology of the image. The objective nature of photography confers on it a quality of credibility absent from all other picture-making. Despite any objections our critical spirit may offer, we are forced to accept as real the existence of the object reproduced, actually re-presented, set before us, that is to say, in time and space. Photography enjoys a certain advantage in virtue of this transference of reality from the thing to its reproduction” [4].

However, how ontologically oriented Bazin might have seemed to be, he ambiguously fluctuated between ideas of reality and illusion implying that “the perfect illusion of outside world” [5] in cinematic reproduction could be “composed of a complex of abstraction (black and white, plane surface), of conventions (the rules of montage, for example), and of authentic reality” [6]. The unavoidable consequence of this was, that the awareness of reality, in the end, could be lost, while watching a film. Thus, Bazin saw illusion as a necessary element in cinema, and further speculated that the spectator’s engagement with the pseudorealistic illusion was based on her existential engagement with the lived reality [7].

The paradigm of indexical image depended on the production process of a cinematographic image during the pre-digital time when there seem not to be a reason to raise questions of content manipulation: what was once filmed, was there on the film, as a proof, as an exact composition of an authentic situation. As film theorist Stephen Prince has brought up, the existing postproduction methods of that time were far more limited. While there were some, they relied on altering the photographic development process, which naturally led to partially uncontrolled results, since this method affected the entire film image. Digital cinema, with its wide variety of control on film footage, meant quite a revolution in this respect. As Prince delineates, “in regard to color timing and the control of many other image variables, digital methods now offer filmmakers greatly enhanced artistic powers compared with traditional photo-mechanical methods” [8].

The emergence of digital technologies within the film industry forced both practitioners and theorists to re-evaluate Bazin’s idea of the realist film image. Media theorist Lev Manovich was one to early dig into the question, what is digital cinema. To briefly summarize his viewpoint, he immediately recognized the ontological shift taking place when the live-action film shot got digitalizedFootnote 1, thus meaning its “privileged indexical relationship to pro-filmic reality” to be lost, when the newly pixelated image became subject to endless modifications, with no respect of origin [9]. Manovich went quite far by suggesting the film image eventually was “reduced to just another graphic” [9]—which is precisely the impression you get while working on a complicated composite shot during the postproduction of the film. In his quest for digital cinema, he completed the following explanation: “Digital cinema is a particular case of animation that uses live-action footage as one of its many elements” [10].

Manovich bowed here to formalism. While for Bazin the film image metaphorically stood for a window into the imaginary world, for Manovich it seemingly represented active framing of the view through a process of selection and combination. Therefore, a single shot within the finished film product could comprise quite contradicting elements, some live-action, some CGI. The result, essentially a collage, also alluded an idea of spatial montage, as “an alternative to traditional cinematic montage, replacing traditional sequential mode” [11]. The central task of digital filmmaking seemed now straight forward. It laid in compositing seamlessly together different image sources, in the manner equal to the temporal montage techniques, providing the coexistence of images in space, like layers on the top of each other, visible simultaneously.

However, as insightful as Manovich’s argumentation is, it does leave one fundamental aspect of cinema almost overlooked, namely the question of movement. Admittedly, Manovich did recognize the difference between illusion and simulation but could not fully articulate the meaning of this division in digital cinema. Instead, he urged digital image—or digital frame—could be categorized as a deceiving impression of space in the sense of renaissance trompe l’oeil. This proposition describes an important function of digital cinema as misleading the perception but does not analyze further, what makes us immerse in cinematic worlds. To fully understand this problem, we might look, how cinema sequences ‘move’ or how they mimic or reproduce the life-like constantly changing situations, and what does this movement stands for.

3 Camera Structuring Space

As opposed to the realist film theories, there have been so-called structural-psychoanalytic premises revered by screen theorist Stephen Heath in his famous essay Narrative Space (1976) [12]. Heath scrutinizes here the essence of narrative space as control of movement required to maintain the story and restrict its single character perspective. In other words, the camera, along with its moving position, confronts by framing the space exploring character, and this way refines the spectator’s possibility to depict these same cinema events from her perspective. He describes this idea in the following way:

“The figures move in the frame, they come and go, and there is then need to change the frame, reframing with a camera movement or moving to another shot. The transitions thus effected pose acutely the problem of the filmic construction of space, of achieving a coherence of place and positioning the spectator as the unified and unifying subject of its vision” [13].

Heath also adapted film theorist Christian Metz’s view of cinema space as “a trick effect”: “if several successive images represent a space under different angles, the spectator, victim of the ‘trick effect’, spontaneously perceives the space as unitary” [13]. Noteworthy, the cinema space prior the digital era gets specified. Despite the availability of various cinematic techniques (i.e., editing, the use of off-screen space), the eventual spatial scheme of the film results from an active reorganization of what is seen in the series of separate shots.

Thus, for Heath, the moving position of the frame in the first place provides a spatial schema required for a narrative to happen, but, equally important, it allows a pattern of observation, thus constituting film spectatorship as participatory. Despite unified as a whole, Heath’s narrative space consists of fragments, or in other words, of sequential shots and eventually frames. It is noteworthy to recognize, that here the concept of space by no means involves the idea of a single shot as a possible collage – or as a spatial montage as suggested by Manovich. As film critic Matthew Croombs points out, “Heath, for all of his claims about the discursive nature of the image, was highly attentive to what was before the camera, and even to the so-called indexical qualities of the image” [14]. Interestingly, Heath’s model of cinematic space creation relies on real camera ‘replacing’ the imaginary witness of events. By doing so, he seemingly implied that the arrangement of time and space in cinema would ‘inherit’ the limitations that cinematography as a craft poses for filmmaking. Although many impressive camera equipment (such as Steadicam used for handheld shots) were not invented [15], when Heath wrote his essay, the pre-digital era of filmmaking was unquestionably labeled by the restrictions in cinematography stemming from the mere nature of physics.

An early indication that the traditional concept of coherent real-like cinema space has started to collapse emerged alongside non-linear editing tools. As Heath has proposed, the movement in cinema relates not only to the movement of the camera but also to the movement from one shot to another. The possibility to try cuts in an uninhibited way, and still be able to return to the earlier versions seemed to create cinematic sequences in which the old bylaws simply did not matter anymore. The cultural critic and post-cinema theorist Steven Shaviro described this change by noting, “there no longer seems to be any concern for delineating the geography of action, by clearly anchoring it in time and space” [16]. The continuity rules are continuously disregarded, eventually leading to combining even mismatching shots. In this “post-continuity” situation, “continuity has ceased to be as important as it used to be” [17]. Shaviro’s account indicates, there is some ongoing discussion, whether the emerging tendency towards post-continuity means that the role of the narrative is “diminished” in the film entirety. However, I would like to think the mind is flexible enough to learn new ways to perceive and interpret spatial schemas suggested by the cinematic movement.

Partially, the development Shaviro describes originates also from the emergence of the CGI. In postproduction, a considerable amount of the total footage can consist of blue or green screen shots. Thus, they do not necessarily include any cues of the geography of the imaginary story space since the character is staged in front of the monochromatic single-color background. When filming on the soundstage with actual scenery or on real locations you are to a great extent faced with the real-world plans of space, but in postproduction, while compositing matte shots, you basically are allowed to choose the placement and the view of the camera and even an idea of camera lens purely based on your inner vision. As a result, the arrangement of space exemplifies something I would like to call a loosened logic of cinematic space, which results in the disconnection of the embodied experience of space.

4 Camera Immersing in Digital Space

To fully understand the narrative space in the digital era, we might now focus on the digital image in its fully immersive power, not only from the viewpoint from practice but also as a space-creating medium allowing the spectator to get absorbed. The most immediate model for this purpose is made available by virtual reality (VR).

There are several ways to characterize VR, however, let us look at the one film critics Thomas Elsaesser, and Malte Hagener have suggested. They describe the variety of VR technologies mainly from a pragmatic viewpoint: 1. VR can simulate real-like environments for diverse purposes, such as learning, training, or therapy 2. VR can make visible such abstract systems that otherwise would be difficult to show, for instance, because they are invisible to eye 3. VR can be used in art and entertainment as a by-product of directly useful applications in the military field, in architectural design, medicine, or the modeling of systems, whose 3-D visualization facilitates the purpose of remote control or tele-action [18].

What is not emphasized here is that VR does produce not only simulative environments but also a virtual camera, through which we can perceive those environments. I believe it is genuinely revolutionizing, to be able to witness film events in any imaginable place, in any imaginable way. This specific quality of digital cinema has started to mold the film making process fundamentally, meaning the whole idea of CGI just belonging to the postproduction phase of the filmmaking process has gradually become outmoded. The ways VR technologies have sneaked into the filmmaking process are all-encompassing. The idea of “100% digital film space” has affected on all cinema practices. For example, the art of production design, which traditionally has been associated with the pre- and production phases of filmmaking, eventually partakes in completing the overall look of the film during the postproduction. The same applies to cinematography. Cinematographers, who as the heads of camera crew were most urgently involved in the production phase of filmmaking, have also recognized their practice turning into “a new kind of unity of art and technology”. Consequently, “the conversion of existing crafts and the activities of cinematography, design, art direction, visual effects, virtual lighting, previsualization, as well as emerging visual practices” forming a new profession of “Cinematographer-Artist-Designer-Technologist” [19].

This is especially true in the new form of film production, namely a virtual production (VP), defined by David Morin, Head of Epic Games’ L.A. Lab, as:

“The ability to mix live footage and computer graphics at once, to get real-time feedback, and to make decisions on set about the VFX and animation. It’s real-time computer graphics on set, where real-time computer graphics can, and do, inform your decisions as a filmmaker. VP is also the process of creating the digital world, beginning with the inception of the movie and ending with the final VFX, centered around the real-time interaction on set. VFX is no longer considered post. The order of production is no longer in order” [20].

Understood this way, virtual production paves the way to the digital cinema as the particular computer-generated simulation, comprising sights and sounds, thus being a real like three-dimensional environment that someone using special electronic equipment might interact in a seemingly physical way. Furthermore, this arrangement differs from seeing a digital image as a representational surface, delivering an illusion of a spatial world. To put it merely, via virtual production, we simulate not just the narrative surroundings, but on some level, the circumstances of filmmaking itself.

5 Enactive Approach

The notion of VP invites us to contemplate further the codependency of movement and space in digital cinema. Here, I aim to refer to the recent cognitive film studies, especially the enactive turn. I can see this turn structuring our understanding of a cinematic experience, which in this case is not just seeing and hearing, but holistically throwing oneself into a simulation of life-like situation. On a common level, an enactive approach describes a human being’s relationship to her environment, especially as an embodied experience: the individual rather lives her environment through the action than experiences it mediated as mental representations. The most basic anchors in this discussion are provided by cognitive linguists George Lakoff and Mark Johnson in Metaphors We Live By (1980) presenting the Conceptual Metaphor Theory (CMT) [21], and by cognitivists Francisco Varela et al. in The Embodied Mind (1991) developing the enactive perspective on cognition [22].

Some revelatory work in film is also presented by Pia Tikka, a brain researcher in neurocinematics and the founder of the Enactive Virtuality Lab in Tallinn. In her Enactive Cinema: Simulatorium Eisensteinense (2008) [23], she explores the psychophysiological grounds of the film experience, and envisions, how the involvement in the world of film is an all-compassing process depending on similar responses arisen while we encounter our daily socio-emotional situations. Thus, a scene of a story can now be understood as a playground of meaningful human interaction, employing the spectator’s specific emotional participation. Much of this view is understood due to the discovery of mirror neurons, which offers a novel view to understand the basis of our engagement with the cinema narrative as the compassion for the film protagonist. Also, film researcher Steffen Hven [24] has offered some valuable viewpoints by considering narrative structures as essentially enactive and experiential. According to him, narratives arise along the continuously evolving affective and intellectual states embodied while watching a film, and thus, can be explored in spatial terms as “our surrounding environment” [24].

Enactivism is also associated with such grand ideas as neuroscientist Antonio Damasio’s theory of consciousness, where the emotion is understood as a response to any change of the bodily state. We see here the core consciousness as a non-cognitive part of the psyche, interacting continuously with its environment and tracking on a momentary basis the bodily changes. The extended consciousness, based on the core self, is the entity that enables the personal identity and memory and makes, for instance, possibly the complex individual goals [25]. As Tikka notes, any cinema experience can be concluded in the Damasian way as “the conscious oscillation between emotional immersion (core consciousness) and back-to-reality (expanded consciousness)” [26].

While it is important to remember that the enactivist movement in its essence is anti-representational, we can complete it with the Damasian idea of mental images. These images are a refinement of the mapping structures and “represent physical properties of entities and their spatial and temporal relationships, as well as their actions” [27]. Maps, on the other hand, “are constructed when we interact with objects, such as a person, a machine, a place, from the outside of the brain toward its interior” [28]. Damasio writes:

“The fact that neurons and brains are about the body also suggests how the external world would get mapped in the brain and mind. … when the brain maps the world external to the body, it does so thanks to the mediation of the body. When the body interacts with its environment, changes occur in the body’s sensory organs, such as the eyes, ears, and skin; the brain maps those changes, and thus the world outside the body indirectly acquires some form of representation within the brain” [29].

This process of body vs. world interaction will lead to naturally emerging narratives, not in the sense that a self-conscious mind tells her story, but in the middle of ongoing events, lives it. As Heath earlier in his category of thinking posed filmmaking as an intentional process, we are here provided an alternative way to approach the process of composing cinematic narratives involving imaginary spaces. The films are created both on the conscious and unconscious level, inherently by manipulating the very same maps—eventually developed into images accompanying reasoning [28]—prompted while enacting the real world. Like Kaipainen et al. suggest, to evaluate the role of the enactive system in content creation, the framework of metaphor theories such as CMT is needed [30], especially when establishing an understanding of how embodied spatial metaphors deliver meanings.

From the viewpoint of this paper, what could this all mean? I feel confident to summarize, from the viewpoint of the enactive cognitive science, the movement in cinema equals the movement of mind. To make humanly sense, any camera, real or digital, must carry the qualities of our body tied existing [31]. Nevertheless, how does digital cinema meet this requirement? Does it mimic the functions of the traditional camera tied to the limitations of the real world – thus reproducing certain cinematic conventions – or does it invent new ways to enjoy unlimited being within a new realm fully?

6 The Movement of the Digital Camera Becomes Cinema

To clarify the question, how do we experience presence in cinematic worlds, I discuss some practical examples of how new digital cinematography aims to simulate intense spatio-physical experiences. I am starting one of the earliest films using CGI. Visualizing Lord of the Rings Trilogy (2001–03) [32] involves quite a variety of digital technologies such as creating computer-generated characters, comprehensive digital sceneries, armies of soldiers programmed to move and behave itself, and making full-size actors appear at hobbit scale [33].

While the range of digital components here is broad, a more important viewpoint concerning digital cinema is provided by the trilogy’s cinematography. Here individually, the CGI scenery has turned out to be a powerful asset in the way it enables the animated camera to express a new kind of movement while exploring the imaginary digital spaces. Thus, the digital environment behind the character remains not a mere moveless background but inherently reveals to possess all the possibilities of digital 3D-animation. The animated camera in the trilogy wanders on some occasions like a bird through the scenery, providing breathtaking moments no physical camera equipment is capable of. In film researcher Kristen Moana Thompson’s word, the camera “thrusts the spectator through dizzying heights, skimming up or down the sides of the two towers, and often moving rapidly from the micro to macro-level or from extreme heights to depths” [34].

Another landmark in digital filmmaking is Minority Report (2002) [35] set in future Washington, DC, in the year 2054. McDowell, who designed the film, has spoken in many contexts of the diverse digital applications developed during the filmmaking process, which in the first place was unusual in the sense that the actual design work was started before the script was even near being completed. McDowell felt it had some extraordinary influence on the eventual narrative, the design consequently representing a strong storytelling aspect [36]. Again, one might easily get overwhelmed by the uttermost detailed look of the scenery based on the almost ‘systemic’ research of the outlook of the future environment [36]. However, the consequences of McDowell’s approach were much more far-reaching. A new film making tool called “previs” was developed, and McDowell describes director Steven Spielberg’s spontaneous reaction:

“He immediately saw that he could direct the prototype of a complex sequence in the same way he would use storyboards, except now he was ‘boarding’ with a moving camera with a prime lense pointing at a low resolution but scale-accurate character traversing designed narrative space, months before shooting.—The same impetus—to be able to direct a camera frame within the virtual world—later developed into the Virtual Camera which now allow a director or key to move through dimensionally accurate designed virtual cinema space in real-time, at multiple scales, with prime lenses, constrained focus, dolly track, atmospherics, and lighting” [36].

Some mind-blowing cinematography was evolved in Minority Report, like in the scene where the main character John Anderton is hiding from the government officials in a large, murky apartment building. An army of spider-like robots are let out to search him, and these nasty creatures crawl through every door crack of the entire building. Based on experiments with previs, the camera brilliantly choreographs through this dynamically complex scene by using the cut-out sections in the ceiling [37]. It also uses embodied patterns to express the role of an above watching eye, seeing through the structures meant to be shielding, being a relentless follower of throughout the long scene eventually stating that I will see everything.

The development is going towards the direction where filmmakers truly wish to interact with the digital content, the emphasis of expression, thus being in providing even more sensual experiences. This means an innovative use of digital technologies emphasizing the embodied-like movement as a storytelling asset, as in Gravity (2013) [38], a film directed by Alfonso Cuarón. Gravity is almost all about the presence, and we have throughout a film a hovering feeling of floating in the space just as weightless as its characters. The actual gravity or the lack of it orients all the action in the imaginary world we are immersed and “all of the physical demands are effectively transmitted to the viewer who experiences immersion in fictional space, therefore empathetically experiencing character’s difficulty in moving or breathing” [39].

The cinematographer Emmanuel Lubezki describes a ravishing moment in the film, which reflects the digital cinema experience almost uniting in the character’s body. The camera’s empathic vision, first perceiving the character within the frame, gradually drifts into seeing the space through her eyes:

“Something very exciting for me to see is Sandra Bullock spinning out of control. We designed all this equipment that allows us to spin the environment around her and give the impression that she is spinning. You can see that in the reflections in her eyes and the visor as the shot is going from an objective shot, where you see her spinning, and then suddenly it becomes this subjective shot, and you start to see what she’s looking at while spinning out of control. I think it’s beautiful!” [40]

7 Conclusion

Ultimately, digital cinema seems to depend on what mind is capable of recognizing as meaningful since there are no restraints that a digital camera can theoretically do. The camera can fly like a bird or flow in the cosmos in a relentlessly complicated way, but at the same time, it balances with the idea, in which manner we can imagine our bodies to move. While the purpose of these existentially oriented functions is solely to enhance presence in the cinematic world, to achieve this, the practitioners are paradoxically faced with an even more complex structure of the film image.

Naturally, in the usual case of filmmaking, the virtual tools are accompanied by the traditional ones, but the pure existence of them seems to have provided quite radical ways to explore the imaginary space. As suggested, the results aim to imply diegetic space structures, even though the filmmaking practices in the digital age are far from using such during the pro-filmic phase. Instead, they rely on the process, which is not only about the film being fragmented into sequential shots but also the shot being fragmented to various assets of diverse origin, layered on the top of each other via parallax multi-channeling. Following, we might even speak of pseudo-diegetic experiences. At the same time, it seems that space continuity is a more complicated question since, especially in post-cinema age, there is a tendency towards lack of cohesion concerning a space represented in successive shots.

As film researcher Kathrin Fahlenbrach has noted, the cinema space is rarely recognized consciously by the viewers. However, at the same time, the interpretation of the narrative is highly guided by the way this space is bodily experienced [41]. The enactive ideas discussed in this paper aim to explore the very basis of this process, where bodily changes lead to the forming of space-related mental images that precede the narrative. The current digital cinema has noticeably advanced in its practice of harnessing interactive technologies. We can see the narratives told by these technologies creating some new storytelling aspects, emphasizing the character’s sensual interplay with her environment. Gravity provides an example here, providing “a minimalist set with maximum effects” [42]. As an intimate film, Gravity’s most ultimate task seems to be in establishing a thickened sense of immersive presence in the film experience.