Untitled Document

Literature and Digital Media: Notes on Theory and Aesthetics

Roberto Simanowski

orcid: 0000-0003-4745-1557

Ladies and Gentlemen, dear persistent friends of digital literature, I am very pleased to have been asked to serve as the keynote speaker for this conference about teaching digital literature. Today I hope to be able to share with you how I see the past and future of this subject and its role within the environment of higher education.

The first observation I would like to make is no observation at all, but a confession, which may or may not surprise and disappoint you, depending on your own relationship to the subject at hand. I have to confess that from the beginning I was no strong believer in the prospect of digital literature becoming the new, exciting literary genre. At least I had my doubts concerning hyperfiction, the first representative of what was later called digital or electronic literature.

Back then, in the end of the 1990s, there were two camps when it came to hyperfiction: the believers and the sceptics. A representative of the first was Mark Bernstein, founder of Eastgate Systems the best-known publisher of hypertext fiction and director of the Electronic Literature Organization from its inception in 1999 to May 2003. I remember well the title of Bernstein’s keynote at the 1999 Hypertext conference in Darmstadt, Germany: "Where are the hypertexts?" Bernstein’s question reacted to hyperfiction’s failure to attract mainstream attention. His answer was at once self- critical and appeasing: He blamed it on the lack of marketing and was convinced the masses would love hyperfiction as soon as they got to know it.

A representative of the other camp, the sceptics, was Marie-Laure Ryan, a prominent scholar in the field of narratology and cyberculture. In her 2001 book Narrative as Virtual Reality: Immersion and Interactivity in Literature and Electronic Media, Ryan calls for a kind of tamed hyperfiction, one that avoids what back then was considered an avantgarde-like advantage of the multilinearity in hypertext: being lost in a labyrinth of text chunks.

While other scholars, such as J. Yellowlees Douglas in 2000 in her monograph The End of Books – or Books without End: Reading Interactive Narratives, celebrated the random, chaotic, and unpredictable narration in hyperfiction as a mirror of life, Ryan held that while the labyrinthine text may be artistically valuable as a unique experience it can hardly serve as a new narrative formula. “in the long run,” she said, “immersive narrativity is much more viable, pleasurable, and diversified than anti-narrativity.”

As a consequence, Ryan defended a more cohesive reading experience through the concentration on “relatively self-contained lexias such as poems, aphorisms, anecdotes, short narrative episodes” so as to “take greater advantage of the multi-media capability of the electronic environment.” Not only did such shift from hardly controllable multi-linearity to carefully designed multi-mediality undermine the premature and, to my mind, erroneous celebration of hypertext as the praxis to the popular postmodern theory of the death of the author, but it also foreshadowed the death of hypertext itself.

When I published my first monograph on digital literature in 2002, I sided with Ryan against Bernstein. I had strong doubts that non- or multi-linear narration would ever have the aesthetic success that enthusiastic academics and their disciples forecasted in the 1990s.

However, this did not prevent me from drawing attention to this new form of aesthetic expression and continuing, with many pioneers and scholars of digital literature, the discussion of a digital hermeneutics in the online journal dichtung-digital that I had founded in 1999 and edited until 2014. Yes, I was skeptical with regard to the nonlinear feature of digital literature, but I never questioned the value of the study of digital literature. I felt then as I do now that it is a subject worthy of academic treatment as one of the many cornerstones to help us understand what today we experience under the umbrella term of digital revolution.

For this reason, I refrained from moving on from the subject of digital literature to the subject of digital art and digital culture without first engaging in thorough discussions of two subgenres of digital literature in my 2011 habilitation Digital Art and Meaning: Reading Kinetic Poetry, Text Machines, Mapping Art, and Interactive Installations.

In fact, I also dealt with the very topic of this conference and invited scholars from different academic institutions in Europe and the United States to share their opinions and experiences concerning the teaching of digital literature. The result of this undertaking, carried out alongside dear and long-time collaborators Peter Gendolla und Jörgen Schäfer from the University of Siegen, was published in 2010 under the title Reading Moving Letters: Digital Literature in Research and Teaching. A Handbook.

Since then other books have been published on the nature of digital literature and the ways of its research and teaching. I am sure you are familiar with Analysing Digital Fiction, edited by Alice Bell, Astrid Ensslin, and Hans Kristian Rustad from 2014 and with Scott Rettberg’s Electronic Literature from last year where Scott rightly makes the claim that electronic literature demands to be read both through the lens of experimental literary practices harking back as far as combinatoric lyric of the Baroque, as well as through the specificities of the technology underlying the production and reception of the works.

As for the rest of my keynote, rather than repeating what I have said in the books I have written and edited on the subject of digital literature, I thought I would focus on an issue that supplements Scott’s plea that scholars pay due attention to the historical as well as technical specificities of digital literature. In my book Digital Art and Meaning I stressed the importance of close readings. Naturally, in the case of digital literature such close readings tend to cover a variety of semiotic elements and include, aside from the linguistic and visual signs, also the actions of a work as well as its interactions with the audience.

Today I want to stress that in addition to the close reading of a single work, we need to read the underlying trend which that work represents. This may be called a media-ontological reading, in that it aims at the ontology of media, something which the father of media studies Marshall McLuhan would have called the message of the medium. The challenge of course is to find the hidden message – which requires both detective flair and intellectual courage.

What this courage means, how such flair may look like, I wish to illustrate with four examples not from hyperfiction but from digital literature, though the more appropriate term in these cases may be digital art. This change in attribution is already part of the point I eventually want to make. My starting point however is, very much in line with the place of this conference, the former colony of Portugal, Brazil, where a special mode of rejecting and appropriating colonial power has been offered based on an element of Brazil that is normally less famous than Caipirinha and Bossa Nova namely: Cannibalism.

I. Eating the Other

The idea of one person eating another often serves as an explanatory model for cultural phenomena in modern times, as is exemplified by bell hooks’s 1992 essay on partnership, entitled “Eating the Other.” hooks’s subject is not the usual metaphors applying to lovers who “simply devour one another.” Instead she’s interested in racist “snacking.” In “cross-racial dating,” hooks contends, white men seek adventures with black women to add the spice of an exotic experience to their middle-class futures with white women.

The Other, which hooks defines as the foreign ethnicity and culture of the partner in this experience, never becomes a fixed part of the men’s lives. On the contrary, the women are merely tried out and come with an inherent sell-by date. The Other is consumed and forgotten. Such snacking is racist precisely because it never progresses beyond snacking.

A decade later, Lisa Nakamura applied this sort of metaphoric cannibalism to the Internet. This time, the subject matter was role playing with identities in cyberspace, specifically white males using female, Asian avatars with handles like Miss_Saigon and Geisha_Guest. Nakamura considers this form of dressing up and making believe with a foreign ethnic identity a form of “empty tourism,” which doesn’t enrich the role-player in any way. It is symbolic appropriation of another’s ethnic identity that is tried out for fun and without any consequences.

hooks and Nakamura use cannibalism as a shorthand metaphor with pejorative connotations. But back in 1928, the Brazilian modernist writer Oswald de Andrade elevated it into a cultural creed in his Manifesto Antropófago. By anthropophagy, he meant a ritualistic cannibalism which he promoted as a way of finding Brazilian identity. Brazilians, de Andrade argued, should neither ignore nor submit to European cultural influences. They should accept, internalize, and digest them with irony, parody, and disrespect.

If we apply the cultural and philosophical idea of cannibalism at work in all three of these cases through the lens of media theory, we can identify various digital media phenomena as forms of “trans-medial cannibalism.” Here the Other is the revered medium of written text, whose symbolic consumption and devour takes place as it is transformed into a post-literate object and a musicalized event.

II. Text Rain and Water Words

One early example of this constellation is Romy Achituv and Camille Utterback’s interactive installation Text Rain from 1999. Individual letters descend slowly from the entire upper surface of a projection screen, and when observers approach the screen they see their own image recorded and projected by a hidden camera in the middle. The installation is programmed so that the letters stop falling when they meet a darker section of screen. That allows observers to use their own bodies to collect, pick up, drop, and catch the letters again.

The installation was particularly successful because it allowed observers to encounter text in such a special way. Suddenly, letters were no longer linguistic bearers of meaning that needed to be decoded, although that was in fact possible since the letters were taken from a poem and formed words and lines, if observers were patient enough to collect them. But of course, hardly anyone thought of putting the letters together and reading them when they could be scooped up with an umbrella and balanced on fingertips. Experience showed that the audience didn’t pay much attention to the text in Text Rain.

Another example is Bit.Fall by Julius Popp from 2006. In this roughly two-meter high and five-meter wide installation, hundreds of valves released drops of water so that they formed letters. Meanwhile, horizontally, the letters formed words. The words were taken from Internet announcements of world events and had a visual life of only two or three seconds. As Popp himself said, this “waterfall of letters” was supposed to express the fleeting nature of what we think is important.

Of course, the irony is that the words disappear before they’ve even truly appeared. From the very beginning, they are not read as meaningful series of letters, but rather viewed as a fascinating phenomenon. We don’t read words made of water. We touch them, we stick our arms into them, we wet our brows with them and we jump through them. Words of water can also be enjoyed from the rear. Like Text Rain, Bit.Fall invites people to encounter text on a visual actionist level. Meaning plays no role in this encounter. The text doesn’t want to be decoded, just seen.

The central, indeed essential characteristic of these cannibalistic text installations is that text is present as non-text and as contradictory presence. The text must remain present behind and specifically in the surprising way it appears, which wows observers.

If letters are replaced by sand, as in the 2001 installation Sand by Zachary Booth Simpson and Ken Demarest, which otherwise is similar to Text Rain in the way it functions and makes observers interact with it, the work loses its anthropophagic basis. Unlike text, sand has no cultural value, and thus as an object of interaction, it cannot appeal to observers as “fallen,” symbolically disempowered language.

III. Pornography and Music

There is something pornographic—in Frederic Jameson’s sense of the visual as per se pornographic—about a text that merely wants to be viewed, not decoded. Like pornography, it encourages enraptured, defenseless gazing. The letters in the text in Text Rain and Bit.Fall appear in such a way that their physical manifestation can no longer be combined into a statement. They do not want viewers to see beyond them to something they describe. They want to be stared at, hypnotically, as people do when they gaze upon a waterfall or a naked body, from which they can’t avert their eyes.

Paradoxically, the hypnotized gaze is also the result of a work that would seem to be the opposite of Text Rain or Bit.Fall. In Bruno Nadeau and Jason Lewis’s 2005 installation Still Standing, letters lying around on the bottom of the screen form themselves into a short, forty-word text in the silhouette of observers, if they remain standing, motionless, in front of the screen for a few seconds. The text criticizes the befuddlement of our time, which it also seeks to correct by forcing those who would read it to stand still and watch. So is the text, in a kind of reverse cannibalism, exploiting this effect in the interest of the venerable cultural technique of reading?

That is the case only for the first glance. Because the rebellion scarcely extends further than its immediate effect on viewers. Would observers pause to watch for longer, if the text were to be constantly renewed and present 400 or 4,000 words to be read?

Technically it would be no problem to follow the eyes of observers and change the silhouetted text before they turn away. But would the text truly have a chance if it were only text? How long can the text extend the immobilization of observers if, as a body that slows itself to be stared at, it disappears behind the world that it opens up? The artists in question don’t press the issue. They keep the moment of standing still and looking as short as possible and in so doing present a text that in its very rebellion kowtows before the anthropophagic effect.

A better metaphor than Jameson’s pornography of the visual is the musicalization of the significant. Music, after all, is never about significance and understanding. Musical sound does not refer to anything beyond itself. It only wants to be perceived in its own physicality. Listening is always staring, which is why in his Critique of Judgment Immanuel Kant classified music as one of the “free beauties” that represent nothing.

This metaphor has been popular since Guillaume Apollinaire noted in his 1913 essay “Les peintres cubists” that cubist and “pure” painting was to previous painting as music was to literature. In this sense, in 1930, Béla Balázs described absolute, i.e., abstract film as “optical music” that doesn’t signify anything, but is instead itself “unmediated materialized significance.” Writing from the same perspective, in 1985 the US art critic Rosalind E. Krauss understood the abstract painting of the early twentieth century as a striving for the “condition of music,” and in 2004 the German philosopher Martin Seel called actions films “music for the eye.” Like color and form in abstract painting, the postalphabetic, anthropophagized text becomes an object to be observed beyond its traditional role as a semantic sign. Its significance is its withdrawal of significance.

IV. Visualizations

The musicalization of the text in digital media reverses the original function of those media. In the 1980s, some people understood the computer as the revenge of the word on television. There were no images, to say nothing of moving ones, on computers, and even computer games played themselves out in landscapes of words. In online role playing, to act was to type. We didn’t stand in front of a house. We read: “You are standing in front of a house.” We didn’t pick up an ax, or have our avatars do it. We typed in: “Pick up the ax!”

The drive toward images—and an initial cannibalistic impulse within the computer’s possibilities for expression—soon expressed itself in the ASCII graphics that were painstakingly put together using the keyboard. Less idiosyncractic, but more important for the computer’s eventual triumph, was the graphic interface.

Since 1984 with the Apple Macintosh, and somewhat later in Microsoft windows, we no longer have to type in a command like this: à“\>copy c:\filename1.doc+filename2.doc a:\trip” to copy two files. We simply click the mouse on the files’ icons and drag them into the folder we want. With Adobe, the graphic user interface was followed by desktop publishing, which rescued computers from the confines of spreadsheets à la Lotus and made them appealing to designers.

The rest of the story is well known. In the 1990s, still images became an integral part of computers. Then came audio, then video. The visualization of communication in the form of photos and increasingly, as Facebook shows, in videos put an end to the revenge of the word on television, reversing the situation completely.

In digital media, too, the “Gutenberg galaxy” and the “literate monopoly,” as theorists Marshall McLuhan and Friedrich Kittler called the span of time in which the book was the undisputed leading medium of society, are giving way to the culture of the visual. The history of media is shifting from a model of sense to a model of the senses.

With images and sounds, the objects are always simply there, whereas text first needs to be decoded. From the perspective of the history of human civilization, this represents a return to an earlier condition of perception at a higher level of technology.

V. Progress as Return

No matter what in fact existed in the very beginning of everything, at the beginning of the word was the image. Before letters, used together, created series of sounds to stand for an object (c-o-wfor cow or d-o-g for dog), they depicted an object. The letter A, which the Phoenicians wrote upside down, imitated an ox: the point was the chin, the horizontal bar the eyes and two tips the horns. Likewise an M was a snake.

The letter visually presented the object depicted much as children acoustically depict a dog when they call it a “bow-wow.” Semioticians would call this visual onomatopoeia and speak of an iconic equivalence of signifier and signified.

Importantly, the sound of an object had no immediate relationship to its representation in writing. Writing did not, as it does today, make language visible. It depicted an object that language represented in its own way: an upside-down A, which in Phoenician designated an ox.

Only once pictography was replaced by letters and words did the visual and acoustic signifiers converge. The ox head was turned upside down and no longer stood for an ox, but for the sound with which its aural signifier began: the A in “aleph.” The technical term for this is acrophony: the reduction of a sound to its top letter.

This clever bit of shorthand was the beginning of various possibilities for combination. As soon as the visual A was no longer an ox head but rather the sound A, it could be used to form all possible series of sounds for all possible objects, if it were combined with other visual signs that also had been freed from the object they originally designated to become acoustic sounds. Written language no longer needed thousands of characters, as today’s Chinese still does, but rather only twenty-five letters, give or take a few, which users could put together as they chose.

Insofar as it increasingly visualizes communication, the computer restores the pictorial nature of the beginning of language. For quite some time, we have no longer needed to describe our avatars on the Internet. We simply display them. In general, we describe what we’re doing less and less. Instead, we post photos and videos. The visual is displacing written language. At the same time, the visual and the acoustic are becoming language.

The abstract process of acrophony of yore was taken to another level by the radical reduction of the alphabet and numerals system of signs to binary code. Just as pictograms and syllables evolve into lettered language, this reduction too has radically expanded the capacity of the remaining signs to describe things. Every image and every sound can be expressed as a series of zeros and ones and be communicated, for example, via the telephone.

No matter what happens on its surface, the computer remains essentially and, for most people invisibly, a textual medium. While the old culture of text, as it was known in the Gutenberg galaxy, may be disappearing, we are increasingly becoming surrounded by unseen text. The anthropophagic treatment of text, not just in art installations but in new media generally, recasts it, in an act of parody, as an invisible presence in the “belly” of the computer.

VI. Reversed Cannibalism

One artistic commentary on this constellation of text and non-text is a roughly one meter high and four meter wide picture made of various colored dots from 2007, bearing the title The Complete Works of W.S.

What looks like a computerized work of abstract pointillism, a visual painting that seeks only to present itself, in reality represents the world of Shakespeare’s writings. The artist, Caleb Larsen, actually transformed Shakespeare’s collected works into differently colored pixels. Here, the text is consumed in anthropophagic style rather than eradicated. Each color is acrophonically connected to the first letter of its name: b for blue, r for red, and g for green.

Bringing Shakespeare’s works—letter by letter, line by line, play by play—yields a collection of colored points that only seems to be random. Observers may simply stare, impressed, at the massive collection of colored dots, but it’s anything but what it appears to be: a radical break from the paradigm of sense in favor of the paradigm of the senses.

The “point” of this anthropophagic treatment of text is that the text survives within the effect the work produces, as Jonah once survived in the belly of the whale. The text doesn’t get lost when it is confined in this way. It can be translated back into its original form at any time with the proper software. Moreover, with a bit of practice, people can actually read the picture, pixel for pixel, letter for letter.

The Complete Works of W.S. would be a clever and a paradoxical answer to the question: what book would you take with you to a desert island? This one picture literally speaks louder than a thousand words. All you would have to do is learn to read the language of the colors without the aid of a computer, which presumably would not exist on the island.

This picture is perhaps the only image that we could say is entirely free of pornography. Staring at it for hours on a hot island day would entail nothing less than seeing through it in a double sense: through the pixels to the letters, and through the letters to a world that exists only inside Shakespeare and his readers. It would be an attempt to reclaim the Other – which has been swallowed by the effect produced by digital media – as one’s own.

VII. Conclusion

I said at the beginning that I wished to share my thoughts on the past and future of this conference’s subject and its role in the environment of higher education. As you realized, I did not provide a canon of works to be taught or thoughts on the didactic of teaching digital literature. This more praxis-oriented take will be delivered by Scott in his keynote tomorrow.

My aim was to broaden our perspective regarding the range of issues that can and should be taken into account when setting out to teach digital literature. My talk, so I hope, illustrates that we are well advised to think outside the box of literary studies and situate our subjects within the new economy of writing and reading and within the increasing shift to a visual culture and a culture of presence, where meaning is replaced by audiovisual events and technical effects.