Digital Wor(l)ds: Using Dynamic Typography as a Mean of Artistic Expression in Digital and Audiovisual Settings

Abstract

This paper intends to carry out an analytical and critical overview of the semantic, aesthetic, and emotional role of the written word and typography in video and digital environments. The paper, in its first instance, will try to describe how written language has evolved from a reading into a visual and interactive experience, but also how it has been portrayed in the creative and artistic fields since the invention of the moving picture, where our focus will be on Duchamp’s film Anémic Cinéma, up until the late 20th century and its emergence in the digital landscape. Later, the center focus of this document will be on the practical application of some of the concepts previously discussed, regarding the development of multiple video installations and virtual and interactive artworks based on the written heritage of Fernando Gonçalves Lavrador, a Portuguese essayist in the fields of Film Studies, Semiotics and Aesthetics. In this section, the prime purpose lies in the exploration of different conceptual possibilities that arise from the usage of typography as a form of expression within the digital and audiovisual arts context, using the texts of Fernando Gonçalves Lavrador as its raw materials.

1 Introduction

(...) ma révolution est dirigée en outre contre ce qu’on appelle harmonie typographique de la page (...). Nous emploierons aussi dans une même page 3 ou 4 encres de couleurs différentes et 20 caractères différents s’il le faut. Par exemple: italiques pour une série de sensations semblables et rapides, gras pour les onomatopées violentes, etc. Nouvelle conception de la page typographiquement picturale [9].

An important feature of printed text has been its ability to serve as a vessel for speech, preserving it for future generations [1]. With the invention of the printing press, typography, the discipline of manipulating text elements on a page (e.g. the size of the letter and typeface, the space between lines, the space between letters, etc.), and the prime focus of this paper, has been seen as the invisible craft, subservient to the semiotic element of the message. But the cries of the Italian futurist, Filippo Tommaso Marinetti, briefly seen in the above citation, have long been realized in a contemporary super-technological context. The importance between content and form can be somewhat disputed in today’s digital landscape, which encourages numerous ways of communicating, be it when one uses only capital letters to express his/her rage or indignity, to use emoticons to express all sorts of emotional status, or to deliver written dialogues in instant messages platforms in high cryptic ways but still understandable between the sender and receiver of the message. In the art ecosystem, producing and consuming text has steadily become a more visual experience, sometimes becoming the prime subject of artistic experimentation - outside of literature, of course. With the invention of the moving picture, printed texts were initially used as key components for transmitting information that the silent image could not convey, but soon after, we would observe the artistic vanguards of the early 20th century starting to experiment with both text and the moving image. In Marcel Duchamp’s Anémic Cinéma, writing would become the central focus of the creation, distancing itself from the theory of the “invisible craft”. And with the rise of the personal computer and artistic digital production the spectator could now interact with the word creating new forms of meaning. This paper will try to briefly analyze these changes around the perception of text and typography, but also how some of the notions described here can be applied in the creation of different video and/or digital objects. These digital artifacts will be produced on top of the written works of the essayist Fernando Gonçalves Lavrador. The objective is to use his texts - which are currently preserved in Cine-Clube de Avanca, a cultural entity that oversees the protection and dissemination of this heritage - as the source of experimentation and then to present the results to general audiences through video and digital installations.

2 In the Beginning Was the Word

2.1 And the Word was Meant to Be Read

As the physical manifestation of a spoken language and a fundamental form for gathering, storing, manipulating, communicating and disseminating information, writing first emerged in Ancient Sumer [3, 5, 10] in the city-state of Uruk - modern day Iraq - between 3200 B.C. and 3000 B.C.. The Mesopotamian cuneiform writing¹ was created for the purpose of long-distance communication performed in the interests of trading. The first writing system is, therefore, created to serve economic ambitions. The Sumerians evolved initially from a method based on tridimensional tokens which translated into units of goods to bidimensional pictographic symbols and later to phonetic signs which eventually resulted in an alphabet with two dozen letters. But although the invention of writing can be attributed to the Sumerians, the first iteration of the alphabet arose from present-day Lebanon, named Proto-Sinaitic or Proto-Canaanite alphabet. With the help of Phoenician traders based along the coast of what is now Syria and Lebanon, this written form was brought to the Greeks who later perfected it by adding vowels. After this transition which would end in an improved transcription of speech, composed of 27 letters, the alphabet would not see any future substantial modifications [11].

An important change in writing’s evolution would occur millennia later, not so much regarding its causa formalis, meaning the shape that the material is formed into [4], but instead in the way we produce it. In the 15th century, with the invention of Gutenberg’s printing press, European societies moved from a scribe-based system, where a few men would manually copy each letter, from a text to another, creating a few pages in a single sitting, to a mass production structure in which one instrument could mechanically reproduce thousands of pages per day. But even if this technological transformation would alter the speed and reach of communication, the main function of written text has mainly resided in its core value to crystallize and preserve speech. This notion is emphasized by Beatrice Warde, in her collection of essays titled The Crystal Goblet [16], who states that typography is, above all, an invisible or transparent craft. Contrary to calligraphy, a form of writing which maintains close relations with the visual arts and that has lost some importance since the introduction of the printing press, the fundamental purpose of typography should be legibility: the ideas of the author are to be transmitted clearly to the reader who, in its turn, should have no effort in understanding them. Since the reader can’t focus on reading the text and viewing type at the same time when truly immersed in its content, in other words, our “mental eye focus through type a not upon it” [16 p.16], typography should be non-intrusive, and not get in the way of our mental voice which continuously produces meaning as it travels through the page lines and interprets the narrative of a text. “The type which, through any arbitrary warping of design or excess of “colour”, gets in the way of the mental picture to be conveyed, is a bad type” [16 p.16]. The author goes even further by excluding the hypothesis that type can have any intrinsic visual and artistic value:

(...) it is mischievous to call any printed piece a work of art, especially fine art: because that would imply that its first purpose was to exist as an expression of beauty for its own sake and for the delectation of the senses. [16 p.13]

The source behind the notions portrayed by Warde, according to Bachfischer & Robertson [1], might have manifested as a response to the different cultural and artistic movements that were coming into existence in the early 20th century, namely the 1920s and 1930s Dadaist and Futurist movements, which were nurtured, in part, by the Bauhaus school and its New Typography² philosophy [15]. This set of events was crucial to free type from its traditional rules and locked grid³. The written form, through modern and visual artistic expression, was now able to move freely and fluidly across the page “resulting in floating typographic compositions, figurative use of letters and fragmented typographic treatment.” [1 p.2].

2.2 Experiencing Text as a Visual and Kinetic Performance Through Film

The liberation of text from the strict guidelines of traditional western typography, and its convergence with movement would be even more apparent with the advent of film and the moving picture. With the birth of cinema, text would become an essential element of silent movies, used mainly to support the images in expressing meaning - when the second was not able to do it solely -, communicating concepts like time, place, or character dialogues. But textual elements were not only an accessory to movies, but sometimes the central focus of the creative work. Such is the case of Marcel Duchamp’s 1926 short film, Anémic Cinéma, which is: “The first film within the tradition of the avant-garde to claim equality of title and image (…).” [13 p.102]. As the movie starts, you can immediately see a subversive attitude towards the prevailing praxis, when Duchamp deconstructs words to create new aesthetic and semantic possibilities. In this case, the result happens by rearranging the two terms that make up the title of the film to form an anagram (see Fig. 1). This is only possible by changing the spelling of anémique - the right way to spell anemic in French -, to anémic, an anagram of cinéma. This purposeful mistake allows him to play with elements of visual composition and quasi-symmetry in the opening sequence.

Fig. 1.

(Source: MoMA)

Title card from Anémic Cinéma, Marcel Duchamp, 1926

The disruptive and experimental view of Duchamp is in line with new forms of thinking that were emerging during the early 1900s, namely the notions brought by the artistic vanguards, like the Cubists [8], for example, regarding the treatment of letters, words and texts, elements that used to be foreign to art, “especially fine art”, as Warde states, and which now took center stage, overthrowing the old barriers that divided literary and visual forms of expression⁴. In this six-minute black and white film, Duchamp shows us new and innovative possibilities of interpreting different language components, by turning them into the protagonists of the movie. As mentioned before, visual, and textual quasi-symmetric compositions are key pieces in the formulation of the title card⁵. This playful act, seen at the beginning of the movie, sets the stage for the 19 compositions that ensue, which are played in rhythmic and hypnotic fashion. These segments portray multiple Rotoreliefs, the spinning discs created by Duchamp that produce optical illusions through centripetal and centrifugal movements, on top of a black background (see Fig. 2). Some of these creations contain circular and elliptical geometrical patterns, while others show spiraling lines of text. Both create an experience of immersion obtained by the whirlpool-like moves, a kind of ballet performance manifested through the multiple pirouettes that slowly pull you in their hypnotizing orbits. This graphic and kinetic act is also accompanied by lines of text that create a sort of word game. Nine discs are illustrated by different sentences in French, impossible to perfectly translate into other idioms due to their nonsensical nature, which according to Geiger [2], are like spoonerisms: “(…) single interplays of homophone words making up nonsense sentences that convey alliterations and rhyme, within the dada and surrealist tradition poetry.” [2 p.110]. These stylistic devices provoke intermittent semantic confusion in the spectator, snapped only by the introduction of entirely graphic discs⁶. The rhythmic switch between text and image panels pays homage to the cinematic experience of the silent movies:

(...) Anémic Cinéma derives from the recognition that by and large the cinematic experience during the silent period was one of an alternation of reading and looking at images in an illusionistic depth (...) for every image there is a verbal passage or between every two images, a title [2 p.110].

The overall experience of Anémic Cinéma is akin to a procedure where the semiotic system of reading literature and the semiotic system of viewing visual art merge to create a new hybrid experience.

Fig. 2.

(Source: MoMA)

Collage of different Rotoreliefs from Anémic Cinéma, Marcel Duchamp, 1926

But even though Duchamp’s only venture into film brings us a new animated dimension to written symbols, the words, and sentences in Anémic Cinéma don’t move throughout each scene. They are fixed on top of a disc which rotates on screen, the actual source of movement. The first instances of movies operating with kinetic typography as we understand it today, originated during the 1950s and 1960s, a period when Concrete Poetry [7] was making a resurgence in the literary scene. It was in the pictures of Alfred Hitchcock that characters obtained motion [1, 6]. Saul Bass’ opening sequences for 1958’s Vertigo, 1959’s North by Northwest and 1961’s Psycho brought us the flying and animated titles (see Fig. 3), achieved by using traditional production methods like multiple exposure or cell animation. The purpose was to “set the stage for the film by establishing a mood, rather than simply conveying the information of the credits,” [2 p. 719]. Although rudimentary, when compared to contemporary standards, these filmic techniques would help to set the stage for more expressive kinetic typography, nurtured, in part, by the rise of new ways of communicating, especially those of cinema, television, advertisement and music videos. By utilizing these original forms of typography, imbued with more expressiveness and motion, “the story is read to us in a particular voice” [1 p. 3].

Fig. 3.

(Source: Paramount Pictures)

Collage of multiple frames from a Saul Bass’ (kinetic) opening title, Psycho, Alfred Hitchcock

2.3 From Viewing the Word to Interacting with the Word in a Digital Environment

“The use of motion on the computer screen changes how we design, how we read and how we interact with typography.” [14]. With the birth and proliferation of computing technologies new forms of depicting the word emerged. In Jeffrey Shaw’s Legible City, the spectator uses a real bicycle to travel through a three-dimensional virtual urban space that simulates famous cities like Manhattan or Amsterdam (see Fig. 4). By using real-time computer graphic technology, these cities are formed by solid three-dimensional letters that shape the words and sentences along the sides of the streets. The textual elements are in conformity with the actual plans and scales of the cities, forming a new architecture of text. In this instance, the interactive dimension of the work is a journey of reading. Traveling through the different streets covered with words, by choosing where to turn and where to go, the spectator is choosing and creating distinct storylines. Legible city creates a bridge between two types of experience, one already discussed in this paper - the reading type - and a new one: the interactive type. By moving forward in a technological sense, further increasing the possibilities of consuming text, one doesn’t need to omit previous experiences with the word. Legible City merges old and new technologies creating a three-dimensional book where the user chooses his own path and story producing personal syntaxes and meanings along the way.

Fig. 4.

(Source: Archive of Digital Art)

Legible City, Jeffrey Shaw, 1989

Other undertakings of text-based digital artifacts can be apprehended in the work of Tom White and David Small. Interactive Poetic Garden, according to its authors, “is literally a fountain of words” [17 p. 335]. Interactive Poetic Garden consists of a physical and immersive installation where water flows down from various cascades into a pool (see Fig. 5). Words are projected on its surface giving the impression of floating leaves. A person sitting on the edge of the pool can interact with the words through a specially designed interface that allows them to interrupt the flow of words, by pushing, pulling, and manipulating their order. The participants treated the work, mostly, as a contemplative environment: some passively watched the steady stream of words, while others would imprison them into chunks to then release them. Speaking about Interactive Poetic Garden, Bachfischer & Robertson explain that

The true beauty of this project lies in the rich and open-ended interaction with type in an unusual form, which goes far beyond the passive reception of expressive typographic performance. If text in expressive typography is interpreted by the designer, text in an interactive environment is interpreted by audiences in new, non-content related ways within a set of possibilities constructed by the designer. [1 pp. 7–8]

Much like Legible City, Interactive Poetic Garden and other new works dealing with language and its written form in a computer-based system, also give focus to the interactive plane of consuming text, enhancing its experience a step further: first from a reading to viewing experience and now to an interactive one.

Fig. 5.

Interactive Poetic Garden installation, Tom White & David Small, 1997–98 (Source: [1])

The interactive principle inherent to the use of typography in a digital and artistic environment is not only obtained by the intercommunication between user and interface, but between the very parts - in this case the textual elements - that form the virtual object. For example, by presenting the right set of conditionals in programming tools like Processing, a language dedicated to new media and electronic arts creation, words can communicate between themselves creating various dynamic events, ranging from beautifully choreographed dances to procedurally generated actions which are randomized, not only altering the position of words on screen, but also changing their size and/or color, in an act that enriches the communicative function of text.

3 Digital Wor(l)ds: A Virtual and Audiovisual Artistic Project

The two titles analyzed are just a miniscule selection of the works that delve into the convergence between text, motion, and interactivity. These serve as the basis for a deeper investigation into the field of audiovisual and digital production using text as the source of artistic exploration. In future studies the focus will be on more contemporary iterations of these dynamics. The goal of this research is to set the pillars for an artistic project which intends to appropriate the textual documents of a Portuguese essayist by the name of Fernando Gonçalves Lavrador to then produce different artifacts that will also explore these conceptual convergences. Next, we will describe with more detail what this initiative will entail.

3.1 The Written Heritage of Fernando Gonçalves Lavrador

In the year 2021, Cine-Clube de Avanca, an important cultural institution of the region of Aveiro, which organizes the Avanca Film Festival, an international audiovisual and artistic annual event, started a project with the intent to create an archive that would present, to a general audience, the memories and work of Fernando Gonçalves Lavrador (1928–2005), an engineer by profession who cultivated a deep affection for cinema and the moving image. He wrote extensively about Semiotics, Film Studies, and Aesthetics, having published various essays on the subject. Nowadays much of his written possessions rest with Cine-Clube de Avanca and these involve essays and books of his authorship, but also of renowned authors, in the fields of Cinema, Music, Art, Aesthetics, Semiotics, etc.; cinematographic and cultural magazines; newspaper clippings; correspondence; drafts; programs, tickets and invitations to film festivals and other cultural events; flyers; and various other objects.

To create an archive, be it physical or digital, one must first develop a lengthy study of the textual materials transmitted from the family of Fernando Gonçalves Lavrador to Cine-Clube de Avanca. In this first stage, one of the authors of this paper was assigned to develop this study, but due to the extent of this collection of documents, which amounts to thousands, our initial goal is to disseminate Lavrador’s legacy through periodic small-scale initiatives. One of these initiatives is the production of multiple video and digital installations that will occur annually during the next film festivals. Although there is a plan to create one for this year’s edition, which will occur between the months of July and August, this exhibition will only be composed of a short selection of objects of the collection. The presentation will also be complemented by a communication about the plans for the construction of the archive and for future exhibitions. Our ambition at the present moment is to produce art installations for the next two editions of the festival in 2023 and 2024.

3.2 Kinetic and Digital Typography: Giving New Life to Hidden Words

The themes which emerge from the stories and history of the character who serves as the premise for this study, meaning a person who dedicated his life for the enrichment of film critics and theory in a literary context, provide the basis for the choices of consolidating concepts intrinsic to text and the moving image. Also, because of the abundant possibilities that the digital ecosystem provides for the construction of new textual paradigms, which can result in the extraction of original and enhanced meanings from the word, the combination of a text-video-computer interface provides a broad and relevant system for the artistic production of artifacts. Hopefully, this can contribute to the further enrichment of this heritage and also assist in the dissemination of a series of documents that currently cannot be viewed by the public.

Besides the theoretical, visual and artistic references discussed in this paper, other ideas are being considered - but still lack an extensive examination - like the field of Digital and Kinetic Poetry [12] or the structuralist and experimental movies of 1960s, 1970s and 1980s: for example Hollis Frampton’s Zorns Lemma or Michael Snow’s So is this, which, much like Anémic Cinéma, challenge “the widespread notion that imagery, whether abstract or figurative is the only visual film resource” [2, p.111]. And since Duchamp’s film is mentioned once again, the figurative possibilities born from the textual compositions of this piece are also being studied. Among them is the notion of aligning text to create geometrical shapes. One of the shapes that presents itself as a relevant metaphor for the object which is being worked on - the written legacy of Fernando Gonçalves Lavrador -, is the curve, more specifically the spiral, because of its conceptual characteristics. This means the notion of continuity implicit in the shape, its constant and infinite movement which travels away from an original point. This, in a certain way, correlates with the idea of legacy that will be studied during the treatment of the objects from Lavrador’s collection. This also creates the right conditions to explore the different designs that form the spiral, be it the Archimedean, the logarithmic or the Fibonacci spiral. And since these artifacts will have a computer base component which allows the manipulations in all-spatial axes, not only can the spiral be explored in a two-dimensional spectrum, but also in three dimensions - another aspect to be worked on is the relation of the spiral with the time axis or, in other words, with its kinetic properties. And much like Legible City and Interactive Poetic Garden, the interactive component is also being studied - not only how to achieve it, but how it can enhance the source materials.

Regarding the technical and material pieces of the artistic development, the desire is to construct part of the audiovisual and digital works using the Processing programming language. Like its definition states, Processing is a language-based system, more specifically it’s written based. For the work to be observed and appreciated on a screen, first it must be typed through computer code language - in this case the program works on top of the Java language. Yet again, from a metaphorical point of view, this choice presents as an adequate and fortuitous one. Of course, the most important point is the wide range of opportunities this program creates. As a quick example, one could record a video with a personal camera, add it to the Processing data system, and then apply a layer on top with a set of written symbols to produce an ASCII video (see Fig. 6). This is a simple quick example, but it perfectly illustrates the capabilities of the program to converge text with figures and then with motion. Through the experimentation of the software capabilities, we intend to discover original ways of creating meaning and, consequently, creating identity and new memories of Fernando Gonçalves Lavrador⁷.

Fig. 6.

(Source: P5.js Web Editor)

Screenshot of a live video taken from a computer camera and rendered with ASCII characters, based on a Processing sketch from Daniel Shiffman

4 Conclusion

The present study is one of the first steps to engage with conceptual, aesthetic, and technical notions that form the preliminary basis for the creation of multiple video and digital installations. Above all, the purpose of using the tools and concepts portrayed in this paper is to start and keep a discussion going around this legacy. A legacy that recently was transmitted to Cine-Clube de Avanca for the purpose of studying it and eventually returning it to the public through a physical and digital archive, which are currently being developed by this institution. In a sense, the works that will be produced will complement what is already being done. From an artistic perspective, this research also creates the right environment to discover the potential of animated typography in audiovisual and digital spaces. How can these settings and a more visual experience of typography enhance and produce meaning from the material that has been handed down to us? How can Lavrador’s identity be perceived through the lens of moving words and thoughts? These questions will be explored from here onwards and the examples analyzed here will prove to not only be a relevant source for inspiration, but a method to solve these challenges. As mentioned previously, the concepts manifested in Anémic Cinéma, like the creation of geometrical shapes from lines of text, but also the possibility to generate word-games from the source material, are being considered. And as for Legible City and Interactive Poetic Garden, these have shown us the opportunities that can arise from the possibility of interacting with the word, and how the public itself can be an important agent in the creation of this unique legacy.