‘The novel as multimedia, networked book’: An Interview with Steve Tomasula
Sandra Bettencourt
CLP | University of Coimbra
FCT Research Fellow

Steve Tomasula

Steve Tomasula is the author of several novels—VAS: An Opera in Flatland [2002], The Book of Portraiture [2006], TOC: A New-Media Novel [DVD, 2009; App for iPad, 2014], IN&OZ [2012]—, short stories—Once Human: Stories [2014]—and essays. His work reflects on language, technology and embodiment at the intersection between the human, society and culture. Inventive explorations of the technologies of the book, his narrative image-texts and image-audio-texts are complex multimodal and multimedia compositions that reveal the interconnectedness of print and digital codes. Expressed through graphic devices and source code, these material interventions are functional elements in weaving a transdisciplinary web of discourses —literature, biotechnology, cybernetics, art history. Heterogeneous and dialogical, the novel-form is transformed into a multilayered media assemblage and a meditation on our post-human experience.

In this interview, Tomasula tells us about the process of producing such complex literary objects, describes the feedback of digital culture on his printed novels, and analyses the specificities of print and electronic media [1].

The process was different for every book (VAS, The Book of Portraiture, IN&OZ, TOC, and Once Human), but in general, I finish the writing before I show a draft to the designers or artists I work with. Layout is part of writing for me, as it would be for a poet, thinking about how lines work together, or how the text will work on the page as a whole, or how an image can carry narrative weight. We’re probably more accustomed to thinking of poetry this way: the way deciding where to break a line, for example, is part of the writing of the poem, though normally we don’t think of this kind of information design as an image. We talk about line breaks and stanzas rather than the white space of the page, but this too is a form of visual writing. I also try to consider all the different materials that constitute narratives as well. For example, the last story in Once Human ends with Hemingway flying over the U.S. As he looks down, he sees American literature evolving, sort of a rewind of literary history. So I wanted to depict the country as a series of maps using ever-more outmoded means of cartography. Another story about a medical illustrator is partly told through the different ways that the same facts, the same body injured in an accident, gets depicted differently by the prosecution and the defense in a court case.

When I have this kind of material and a draft in hand, the designers and I begin our discussions—it’s a lot like a playwright talking with a set designer about how to stage a play. After they’ve had a chance to work with it, we keep meeting and talking, often online. Most of the time they come back with ideas that are great; sometimes we work through issues of translation or interpretation. For example, in VAS, I had laid out a page with nothing but the formula for salt, NaCL. The designer laid it out as a block of solid text, which seemed very serious while I was thinking of it as kind of a joke—funny, but serious at the same time—which is why I’d tapered the margins on both sides so the page would evoke both a Greek column, and a woman’s figure—I wanted it to be a visual pun for Lot’s Wife, who was turned into a pillar of salt, but told in the language of chemistry instead of scripture. It’s important to work with designers who can get those kinds of cross-genre in-jokes, references, or puns, who have a shared aesthetic, or can make literary as well as visual connections.

TOC was very different in that there were over 15 people working on different parts of it, none of whom saw the entire work until it was done (with the exception of myself, of course, and Chris Jara, the programmer putting all the pieces together). The text had to be finished first because once it becomes part of an image, it’s very hard to make changes in a work where text is imbedded in images: adding a comma, for example, might involve changing the artwork on 30 different screens. The writing was actually done about 10 years before TOC was published. In between, what I mostly did was art direct, or coordinate the work of these dozen people, working in several countries. It was like getting a band together where you need a drummer, someone on bass, someone to sing… For example, Tim Guthrie made the sequence where a blue marble, like a raindrop, falls into a sea and the splash morphs into an island. To make this sequence, I sketched out a storyboard; Chris Jara created a digital model of the raindrop, which he used in the previous animation, the animation that would cross-fade into the animation Tim was going to make. That is, I gave the last frame of Chris’s animation to Tim to use as his first frame. I also supplied Tim with his last frame: an oil painting of the island made by another artist. He created the animation between this first and last frames, digitally painting on every frame he created to transition from a digitally-made to a hand-made look. While he was doing this, the music was being composed, performed, recorded. Voice actors spoke the lines. Then in the studio we brought all the pieces together. This had to be combined with the programming, since it was interactive. As you can imagine, there was a lot more of a scattered approach than “work flow.”

For VAS, I created a rough-draft of the layout in WORD. I’d print out, and spread out the pages on the floor to see how the pages worked together visually. Near the end, when I was working on whole sections or passages of the novel, the whole floor would be covered and I had to stand on a ladder to get a bird’s-eye view. The final layout was in Quark (software that is now antique!). For the other books the final layout was in InDesign. I write in WORD and layout what I call the 1st draft in InDesign because it’s easier to move images around (though of course there are box-loads of text drafts and notes and sketches in between the start and this “first” draft).

TOC was the most tech heavy: I made some of the rough drafts using software like GarageBand, Photoshop; iMovie. Director was the main program used to put the pieces together. 3D pieces were made in Maya, and LightWave; other pieces were made with the usual software you’d think of: video and audio editing in Final Cut Pro, for example. The clock that runs in several of the animations is actually an oil painting into which 5 other animations were fit: so it’s a composite of multiple animations, assembled in Final Cut Pro. We also had a way to use a commercial rending farm at night after everyone in the office had left for the day.

Your question reminds me of Katherine Hayles pointing out that all works are digital, if their composition, publication, and delivery are taken into consideration. Yes, digital photography, WORD, scanners, Photoshop, etc. All of the usual digital authoring tools were there from the start. The Internet was a huge help in terms of locating a data bank that had the gene sequence that appears in the book, for example. The first time I downloaded it, it filled up my hard drive (this was back in 1997 or so). But other than WORD, the tool I use most is probably a pencil. Either sketching, or especially editing, and rewriting. Really, I think it’s most important to focus on the narrative and bring in whatever tools are needed to tell the story, though I realize that many e-authors begin with the technology and think about what story they can tell with it. I’m thinking here of e-authors like Nick Montfort or poems where the text and code are equally visible, and equally important.

I hadn’t thought of this till you asked, but looking back on it, all of these considerations entered in at different points during the writing. It’s actually hard for me to distinguish between the ideas, the media, and language. That is, I try to make the materials part of the narrative so I think these distinctions blur very quickly. I think you can see what I mean if you imagine how the narratives we construct as a society shape who we are: laws, for example, that determine who has power over another. And these narratives always include the ‘materials’ out of which they are constructed: images, be they scientific charts, or photos, or network diagrams based on a phone company’s metadata. Imagine, for example, depicting a person in a watercolor painting; now make the same portrait using a surveillance camera. Or data points gleaned from their charge-card purchases. Each of the portraits would tell different stories about the person because they operate under different compositional constraints, different systems of logic. The “material” that the portraits are made of bring in different associations. The same is true with books: the medium is inseparable from the message, as Marshall McLuhan first proposed, or at least it’s an inherent part of the message. It’s impossible to talk about a medieval manuscript without thinking about how it embodies a whole way of life, with its pages made of animal skin, its colors made of charcoal or berries, the pen used to create it plucked from a goose—the letters written by hand—manu in Latin or mão in Portuguese—where we get the English word “manuscript.” All of its materials are part of the natural world, just as the ebook is an expression of our time, with its global economy, databases, conflict minerals…. The materials become part of any story read on it. A person reading the Iliad on a Kindle is not reading the same story as one who reads it in a mass-produced paperback book, or clay tablet. So really, the ideas of page navigation, or multimodality and the rest emerge from the particular story being written rather than imposed from outside, even if they are all interrelated, as you say. Maybe I try to bring the materials to the surface more than they are in more traditional books?

For example, VAS: An Opera in Flatland is a novel about the biotech revolution; it explores the way we are able to manipulate genes the way we manipulate text. So I tried to make the book a metaphor for the body to mirror how the body is beginning to be seen as a kind of book: both can be edited; we copyright both words and genetic sequences, the letters that make up each language. I spent a lot of time considering how a reader would navigate its pages—how I could encourage her or him to read straight down or across a page, for example. Or feel free to assemble phrases in ways that would suggest different meanings the way genetic letters can make either an oak tree or a mouse depending on how they are assembled. Late in the novel there’s a multi-page collage of examples from bio-engineering, body modification, etc. I wanted to give readers permission to skip across the collage so I kept a line from a song that is part of the narrative, in bold along the top of the page as a way to indicate that it was okay to keep turning pages; my hope was that a reader could appreciate the density of the collage—seeing it as a visual about information overload—without feeling compelled to actually read it, though a reader could do that too, of course. I tried to indicate that the collage was like footnotes that someone could read for more detail, or background, but didn’t necessarily have to follow the main story.

The Book of Portraiture is sort of a history of representation—it starts with the invention of writing, then skips across medieval Spain, a 19th century case history of “neurosis” to the present era of surveillance, Big Data, and genetic engineering. So I tried to evoke this history, this archeology, with the paper: If you close the book and look at its edge, the chapters look like layers in an archaeological dig. In other words, the multimodal aspect of the book is in the DNA of the stories; they couldn’t be told without it. Or at least they would be different stories without it, the way a movie based on a novel is a different experience.

TOC is a novel about time, and so I wanted to use time as one of the materials of that story—which meant it would have to be told through some time-based medium which led to it being created in a form that was tied to a clock: the clock of the computer it was read on. Here, the idea of page navigation came early on, before some “pages” were written, because the passages were going to be linked, and other elements of the programming flowchart had to be worked out. That is, I tried to use the clock of the computer as part of the story just as ink and paper were part of the story of VAS. For example, the rate of change in an image that the clock of the computer could allow was a big issue early on: we wanted the astrolabe to turn slowly, like the hour hand on an analog clock, or the shadow on a sundial, and 20 years ago that was a real challenge. So, yes, from very early on I was thinking about the narrative in terms of storyboarding, and flowcharts; the navigation, what parts would be told as images; which parts did readers have to read for themselves; which parts would be read to the reader as an audio track, what kind of voice or music the audio could have; when a reader would scroll, when a reader would click, what would happen when an image or word was clicked on—where the action would send the reader—when a reader would be allowed to repeat or fast-forward, and how to make it all work on screen, etc.

TOC is read in real time, i.e. the time it takes a reader to read the novel; but it also invokes other experiences of time:  mythic time, geological time, personal time, cosmic time— I wanted to get across how Time, in all of its versions, is a construction. Camus once said that every novel was an image of a philosophy. In that sense I was trying to create narrative or visual images of different conceptions of time, and how they become articulated in our lives, how they shape us as a society, but also as individuals. I was trying to explore through fiction how time itself is a fiction. Not to say it isn’t real, or powerful. In fact, it shapes us in the way that geographies (or stories) shape us. It’s a fiction based on lived experience. Think how our conception of the self has changed once the story we told about time changed. Once the invention of the lightbulb further loosened our dependence on night and day, or railroads made the invention of time zones necessary. Or once enough of us began to believe that the earth is much, much older than the 5,000 years claimed in the Bible—or more recently, the growing awareness of how briefly humans have been on it: an aberration, really, with no guarantees that we will continue.

So really, I think finding a form for the narrative drives the media rather than the other way around.

Yes, and no. Early on, when the Internet was fairly new and the web was text-based, I used email a lot, especially for research, contacting medical libraries, or contacting labs to find places that would let me visit; things like that. At one point I found a list-serve of airline mechanics who used it to share information about different planes. I ended up in a long discussion with a captain in the U.S. air force about how a mechanic could rig the wiring harness in a jet to make it crash, but look like an accident. He was stationed at an airbase in Alaska, so I guess he didn’t have anything else to do other than talk to me! (This was obviously before 9/11. Someone would probably end up in Guantanamo if they tried to have that discussion today.)

Back then, the designers and I shared files by mailing CDs, or SyQuest disks, which were the massive storage devices at the time (a whopping 44MB). The ever-cheaper cost of memory, or something as simple as Dropbox, or the ability to share desktops on Skype, has obviously transformed collaborative work—made it so much easier to do—one of the reasons there is so much more collaborative writing now. The novel I’m working on now follows our changing conceptions of nature, especially the idea of extinction from the time when most people didn’t believe in it till now: a time when we can’t avoid it: extinction is now deep in the fabric of our personal and social lives. And so, yes, FB and other social media will be built into the last chapter.

O While writing TOC I was very conscious of what’s now thought of as “Classic E-Lit,” Hypertext novels—like Michael Joyce’s afternoon, a story or Shelly Jackson’s Patchwork Girl. I tried to learn from their structure but also tried to think about the differences between literature on screen and literature in a printed book. I wanted to take the book as a medium seriously, which meant doing things with the print book that couldn’t be done online, and things with the electronic novel that were impossible in print. Otherwise the differences seemed to be more about distribution, publishing, and sales than art or literature.

One quality of early hypertext novels was that they never seemed to end—words were linked to words were linked to words, forever…. Sometimes this quality contributed to the story told: Stuart Moulthrop’s Victory Garden is partly about the way America’s Gulf War never really ended, and so a book that never really ends helps tell this story (and it’s shocking how relevant both the story and the form of this story are today). But because all the early hypertext authors were using the same authoring tools (Storyspace) the books all seemed to have similar forms, and so, in terms of form, a thematic similarity. It was also impossible to tell where you were in a novel, or when you were trapped in a cul-de-sac of text that you’d already read. These were some things I tried to avoid. What most drew me to these works, though, was how beautifully written some of them were. A novel like Michael Joyce’s afternoon, a story would have stood up as Literature with a capital L whether read in print or on screen. I think many e-lit authors get caught up in the ‘e’ part of e-lit. And readers too: focused on the technology instead of what it’s being used to say. But I’m most drawn to those e-works that emphasize the literariness of the literature—and try to write that way—with the electronics in its service.

Back when I began writing TOC, I worked as a writer for an electronics manufacturer, so was in research labs a lot. I remember seeing a new technology that engineers had developed to store images on a floppy disk—a first, simple storage device for digital images, I believe—and at the time this was amazing. This was back in the 80s. The idea of putting that technology in the service of a novel was powerful to me. It was apparent what kind of hybrid novel could be written with a tool like that—the way children’s books used pictures as part of the narrative, only on steroids. At the time though, it was hard to read novel-length amounts of text on screens. I don’t know if we hadn’t been trained to look at screens for hours on end yet, or if it was because the resolution wasn’t that great and it was physically hard to look at a screen. Critics of e-lit used to say reading on screen was like trying to read text printed on a lightbulb. In any case, I had in my mind the idea that people couldn’t tolerate lots of text on screen, so I tried to write screen by screen; I wrote on notecards, figuring that the amount of text that could fit on a notecard was about as much as could fit comfortably on a screen.

In any case, I had always thought of TOC as a book. Not a work of e-lit—for me, it was a novel—where the most important material was its language. But it was also a book that could do things a print book couldn’t: it could have words that danced on the page, set to music. How exciting to think of a book as a multimedia experience the way theater or opera were multimedia experiences. In fact, when I go to the opera I love to sit up high enough so I can look down and pretend I’m holding the stage in my hands—like a book. And I wanted readers to have a sense of holding TOC in their hands as if it was a tiny opera stage that they could hold like a book. A chamber opera with an audience of one is how I thought of the relation of the reader to TOC. Unfortunately, the technology needed for a number of the visual effects I wanted to use to tell the story wasn’t really there yet. Or rather, you needed a Hollywood studio to create them. Then the technology got better and it was possible to create the same effects on a commercial workstation, and we are able to publish TOC as a DVD. It really wasn’t until the iPad came out that the device I had always imagined TOC to be played on was finally invented. You can see some excerpts at www.tocthenovel.com.

Definitely! One critic writing about my work said that I treat the book as a platform, and I really like that idea: thinking of the printed book or the electronic book as just different platforms. Or OSs that are suited to some narratives better than others. While I was writing VAS, electronic literature was coming on as a force, and the book was very expensive to print so the question of publishing electronically came up. But I wanted VAS—and also The Book of Portraiture, but especially VAS—to be real objects in the world. I didn't want them to be virtual. As I mentioned earlier, if VAS was a palindrome relationship to the human body it had to have a body, body text, and a spine. Further, if fiction is real, I wanted the book to be a real object in the world, not a fantasy a reader gets lost in, i.e., a vehicle to escape the world. So it was printed in a way that couldn’t exist online. The hardback edition has a book jacket that lets its cardboard backing show through, for example. Then there is a layer that most books have, but it’s printed like muscle. And finally the spine is covered in a fake, flesh-like leather. That is, there is a bone, muscle, and skin layer to the hardback cover—i.e. the materials are part of its context from the moment the book is picked up. Inside, there are lines of music that are too fine for the resolution of a screen; they could only be printed by a high-res press on fine-grained paper. And of course I had in the back of my mind all those other print books that used images, layout, info design as part of the narrative: Johanna Drucker’s amazing artist books. Or Lee Siegel’s Love in a Dead Language. Conversely, TOC is a novel that couldn’t exist in print.

Just including an image in a novel increases the complexity of publishing immensely. And also reading. Including images, or changing page sizes, introduces many technical difficulties, copyright issues, and they all drive up the cost of the book, often to the point where they are prohibitive—especially if the print run will be more like a book of poetry than a bestseller. The only way VAS could have been published was for it to be printed in China, where printing costs were about half what they were in the U.S. All of this introduced huge delays, in terms of exchanging proofs and corrections, language differences, etc. The manual for how to print VAS was about an inch thick. My story collection Once Human came out almost two years late for the same reasons. Including images, even simple ones, means including other people, skills, materials, technologies, costs. It involves working with the printer more closely than you have to with text-only books. I think things are changing now, but there’s also resistance by some readers to hybrid fiction in general.

Given how multimodal something as pedestrian as a newspaper can be, it’s very odd how many people still think of literature as only text. I understand that even Sebald had to argue with his editor to include the few black-and-white photos he used in The Rings of Saturn. And as great as that novel is, some critics still dismiss the photos as superfluous—which maybe they are, for the kind of reader who isn’t able to read a visual image—or see how his photos contribute to the melancholy mood of his novel. Somewhere in the back of my mind I had an idea that some people would come to a visual novel for the art while others would come to it for the writing. But I think what happens instead is that the audience gets narrowed for this kind of work because it requires a reader to be as visually literate as they are linguistically literate: it requires readers to be equally sophisticated with images and layout as they are with text to see how they work together to form a whole. My first editor was also a poet who collaborated with video artist Gary Hill; he published books on visual art as well as authors like Blanchot. He understood immediately what I was trying to do. I can say similar things about the other editors and publishers I’ve had. I understand that the books do look different from “normal” novels. And the editors I’ve had have had to explain them to boards or get them past marketing departments. I’ve been lucky to have editors and publishers who are willing to think differently. (This is partly why my novels have the subtitle ‘novel’; an editorial board member wanted the word included in the title so people who came upon it in a bookstore would be able to tell what it was.)

This cuts right to the heart of the matter doesn’t it?—with all of the authoring tools now available, and the publishing channels that are opening up, we’re all forced to rethink basic things that authors, readers and publishers only a few years ago could have taken for granted: what it means to “write,” what a “book” is, what it means to “read.” These are exciting times for both readers and writers, especially given how the walls between readers and writers are collapsing; literature today is sort of a wild, lawless land in the way everyone is trying out different ways to live as a writer in an ongoing, and rapidly-changing practice. They’re trying to reinvent, or reconceive publishing, and reading. I’m reminded of a statement that the composer Morton Feldman said about visual arts in the 50’s: ‘for a brief time, no one knew what visual art was, and that’s why it all happened,’ by which he meant why earth art, performance art, conceptual art, and all the rest that came along after people began to think of art as more than paintings that hung on walls. 

And yet, painting didn’t go away—it just became more painterly—just as movies didn’t kill off theater… That certainly seems to describe publishing and codex books today, and publishing is always co-author to what counts as literature. So I think born-digital works, video-poems, machine-writing, and other kinds of non-traditional writing are making us ask exactly this—why write text?—why publish a book as a box of words?—and I think one answer will be some version of what painters gave at the advent of performance art, or playwrights discovered as movie-making matured: that printed text, too, is a medium: a really great medium, that can do some things better than other mediums, and every art form is always, no matter what else it is about, at least a little about its own medium. Maybe we haven’t yet worked out what it is that printed text, or a printed book, does better than other mediums, but it seems to have something to do with the meditative pacing it invites. The physical object of a paper book, the way a reader interacts with it, and can become engrossed in it—that cliché of being lost in a book—or lost in thought even if the book calls attention to itself, or its story, as metafiction does. Just the physical act of turning a page slows down reading, and this pacing seems to encourage the one-to-one intimacy that a reader can have with a book. It seems to facilitate rereading passages, which is so easy to do with a codex that the effort is practically invisible. It facilitates a more meditative mindset, or reading experience—something that’s important in a word-image novel where it’s important to sometimes just stop reading text and look at it—to see how some images, or words echo others—The ability to have a physical sense of where you are in a book, even the heft of a book gives a reader a sense of its narrative scale. It gives a sense of a narrative whole by which to understand the part. These are things that seem diminished when reading on a screen, since all screens are connected to the Internet. I also think the physical codex has an effect on reading long after the book is closed: just seeing it on a shelf makes me think about it, remember and reconsider it in a way I don’t do with a virtual book.

I’m far from the sort of purist who thinks that a person couldn’t read these books electronically, though. In fact, I often “read” the same novel across platforms: as an audio book; print book; and Kindle book. But I recognize that they are different experiences: the audio book continues whether or not I’m paying attention; the Kindle reads me as I am reading it: if I mark a passage or take a note, my actions are reported back to Amazon (a Kindle shows me, for example, what passages other readers have marked). So there’s this constant sense of social reading on a screen—the way watching a film in a theater is a social experience—the way reading an online newspaper inherently contains an act of voting for particular articles. Every time we click a link, an algorithm is feeding that action into the calculus that will help determine what will be written about in the future, or shown to me as a reader—what ads will be displayed on my screen. I’m reminded who the publisher thinks I am.

Every medium, and I do think of print books as a medium, carries with it genre expectations. We normally don’t go to a film expecting to participate. With a print book I’m not networked so feel more focused; more inner-directed, more able to stop and think which is different from the kind of “interrupted reading” on screen where I’m constantly tempted by links (and it’s hard to not click on them); it’s too easy to channel surf—or check email or FB.

I’m not sure if this is a worse reading experience, but it is different, and one that seems to work against the experience, and thoughtfulness many novels try to invoke. Maybe this doesn’t matter so much for those novels that are more like entertainment—novelizations of movies or TV or bestsellers—and so aren’t damaged by ‘distracted reading.’ But I do think it works against the kind of novels that are more language intense, that ask readers to participate in being coauthors to meaning, that ask for a certain amount of meditative space—it works against those novels that are hard to read not because they are difficult, but because part of their pleasure is stopping and thinking about what’s being read. And yet, reading itself, the act, as is publishing and writing, is in constant flux. To the medieval, ‘reading’ a book meant memorizing it. The idea of scanning a text for information would have been alien to them (and probably impossible given their lack of punctuation, or even word spacing). The birth of every new media is said to be the death of the old but all that happens is that it makes the old technology rediscover what it is about it that makes it special. I think e-readers are making us rediscover what it is that is special about print, and its disconnect from the on-line world seems to be one of these things that facilitate ‘slow’ or more meditative reading.

At the same time, it’s very exciting to see the birth of a new genre: the novel as multimedia, networked book—a narrative form that seems particularly in synch with our multimedia, networked moment.

Notes

© 2016 Sandra Bettencourt.
Licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 International (CC BY-NC-ND 4.0).