Here is the full text of my talk from the 2015 conference of the Society for Literature, Science, and the Arts — part of the panel on Video Games’ Extra-Ludic Echoes (organized by David Rambo, and featuring talks by Patrick LeMieux and Stephanie Boluk, David Rambo, and myself).

On the “Parergodic” Work of Seriality in Interactive Digital Environments

Shane Denson (SLSA 2015, Houston, Nov. 15, 2015)

I want to suggest that popular serialized figures function as indexes of historical and media-technical changes, helping us to assess the material and cultural transformations that such figures chart in the process of their serial unfoldings. This function becomes especially pronounced when serial figures move between and among various media. By shifting from a medial “inside” to an “outside” or an in-between, serial figures come to function as higher-order media, turning first-order apparatic media like film and television inside out and exposing them as reversible frames. But with the rise of interactive, networked, and convergent digital media environments, this outside space is called into question, and the medial logic of serial figures is transformed in significant ways. This transformation, I suggest, is not unrelated to the blurring of relations between work and play, between paid labor and the incidental work culled from our entertainment practices. In the age of transmedia, serial figures move flexibly between media much like we move between projects and contexts of consumption and production. The dynamics of border-crossing that characterized earlier serial figures have now been re-functionalized in accordance with the ergodic work of navigating computational networks—in accordance, that is, with work and network forms that frame all aspects of contemporary life.

I will come back to this argument in a moment and elaborate with reference to Batman and his movement from comics to film and video games, but first let me say a few words about “the work of seriality.” In the nineteenth century, production work became increasingly serialized as it was fragmented and mechanized in factories, culminating in the assembly line, the paradigmatic site of serialized production, and eventually leading to digital automation and process control. Ultimately, of course, this trajectory was spearheaded by capitalism’s own seriality, or its structuration as an endless series of M-C-M’ progressions. At the same time, works of culture also fell under the spell of the series. The industrial steam press churned out penny dreadfuls and dime novels, before comic strips, film serials, radio and TV series took over. The simultaneous rise of serialized work practices and serial “works” of culture is too massive, I suggest, to be a sheer coincidence. And concomitant with these two was a third form of serialization: that of cultural identity or of subjective experience itself. As Benedict Anderson and Jean-Paul Sartre before him have argued, new forms of community, identity, and perception were based in the serial work of media, such that, for example, the serialization of daily papers, consumed more or less simultaneously by an entire nation, could produce the nation itself as an “imagined community” of serialized subjects. Anderson’s conception of the serialities of nationhood or the proletariat suggests a material connection between the minute level of concrete serial media practices and the broad level of discursive, cultural, or imagined realities—a connection that I want to pursue into the realm of digital media.

I want to suggest that serialized media are able to leverage these shifts in the nature of work/works because they function according to the logic of the “parergon,” as described by Jacques Derrida. Etymologically, the term parergon is composed of the prefix “para-” (next to, or beside) and “ergon,” which derives from Greek for work. The parergon is thus literally “next to the work,” marginal or supplementary to it, as a frame is with respect to a painting (or an hors d’oeuvre with respect to the main course of a meal).

But the picture frame in particular demonstrates an essential reversibility: on the one hand, the frame serves as a background for the work, as a ground for the image it frames and selects or presents. On the other hand, the frame can also be absorbed into the figure when seen against the larger background of the wall, as when we take a broad view of a row of paintings on a museum wall before selecting one to observe more closely. The frame is therefore subject to repeated figure/ground reversals, and it’s the same with serialized media, which are constituted in the flickering interplay between an ongoing sequence and its articulation into discrete segments.

A serial figure like Frankenstein’s monster embodies this interplay and mediates it as a higher-order reflection on media change. The monster is of course part of a film’s diegetic universe, for example, but it also exceeds that frame and partakes in a plurimedial series of instantiations. We never just see Frankenstein’s monster; we see an iteration of the monster that stands in extradiegetic relation to Karloff’s iconic portrayal and to a series of media and mediations of the figure. And we should not forget that Karloff’s mute monster, which contrasts sharply with the eloquent monster of Shelley’s novel, once served to foreground the transition from silent cinema to the talkies. Figure/ground reversibility is an essential precondition for plurimedial seriality as such, specifically enabling the foregrounding of mediality that allows the serial figure to serve as a figuration of media change.

But with the rise of digital media, the formerly discrete media across which serial figures were deployed come to mingle in much closer proximity. What Henry Jenkins calls our “convergence culture” responds by coming up with new ways to tell stories (and to sell commodities) that take advantage of the coming-together of media in the space of the digital. In Jenkins’s version, transmedia storytelling is inherently serial, but much less linear than a conventional television series might be, as it allows the reader/viewer/player/user to explore various facets of a story-world through movies, games, textual and other forms, allowing for a variable order of consumption that corresponds, we might say, to the database structures in which digital information is stored and (interactively) accessed. But transmedia storytelling often aims to smooth over the disjunctures between media installments; the parergonal logic of figure/ground reversals that sustained serial figures and allowed them to track and foreground media changes is thus transformed. A serial figure like Batman thrives in this new environment and traces this transformation in relation to computational mediation and the shift from a parergonal to what I call a parergodic logic.

With the term parergodics, I want to link Derrida’s notion of the parergon with Espen Aarseth’s use of the term ergodics to describe processes and structures of digital interactivity. Ergodics combines the Greek ergon (work) and hodos (path), thus positing nontrivial labor as the aesthetic mode of players’ engagement with games. For Aarseth, the arduous or laborious path of ergodic interactivity marks a fundamental difference between digital media such as video games or electronic literature on the one hand and traditional literature and narrative media on the other. For whereas the path of a narrative is fixed for the reader of a novel or the spectator of a film, it must be generated in digital media through a cooperative effort between the user and the computational system. The signs composing the text of a video game—including textual strings, visual perspectives, narrative and audiovisual events—are not (completely) predetermined but generated on the fly, in real time, as the player makes his or her way through the game. Ergodics, the path of the work or the work of the path, therefore describes the nontrivial labor at the heart of gameplay. But to expand this beyond Aarseth’s narrower frame of reference, the concept of ergodics can also be seen to ground a wider variety of interactive and participatory potentials in contemporary culture, where computational networks are implicated virtually ubiquitously in entertainment, social life, and work. The borders between these realms are remarkably unclear (think of all the things people do on social networks and the virtual impossibility of distinguishing clearly between work activities and play), and it would seem that this has something to do with the indifference of computational media to the type of contents processed. This computational indifference to the phenomenological modalities of human experience – or to the differences between the analogue media that at least partly corresponded to those modalities – leads, as Mark Hansen argues, to a divergence between mediation in its classical, perceptually oriented form and a new form of mediation that channels human affect into the process-oriented project of establishing ever greater networks of pure connectivity. This is the larger significance, I propose, of what Steven Shaviro calls “post-cinematic affect”: in contrast to the cinema, which was constituted by the storage and reproduction of perceptual objects, ergodic mediation involves acts of affective interfacing with the fundamentally post-perceptual realm of computation, which is algorithmic, distributed, and nonlocal, in contrast to the phenomenological basis of human embodiment. Clicking on a Youtube video not only delivers perceptual content to your embodied eyes and ears, it also delivers computational content – information about affective, epistemic, and monetary valuations – to the routines of network-constitutive algorithms. In this environment, play activities not only involve the execution of nontrivial work, as Aarseth argues, but corporations and financial interests, among others, continually find clever ways to disguise work as play, to “gamify” our labor, both paid and unpaid, while mining the data generated in the process in order to profit from both dedicated and incidental work. In this environment, as Matteo Pasquinelli has argued, virtually any investment of attention or affect will also generate a surplus value for Google, Facebook, etc. – a value produced and accumulated parasitically, without regard for any significance we may attach to the contents of our digital interactions, by means of computational algorithms functioning on an altogether different level than the human concerns that feed them.

As a result, media “contents” become incidental or marginal to work, so that our so-called “participatory culture” might better be termed a “parergodic culture,” where cultural “contents” are reversibly supplemental to the nontrivial labor of interactive work. But the notion of “parergodic culture” suggests also that there might be para-ergodic margins from which to witness the shift, to take stock of it in the process of its occurrence. This is where the parergon meets ergodics, and it’s in this reversible margin of parergodicity, neither completely inside nor outside the realm of ergodics, that I’d like to situate the serial work of Batman from about the mid 1980s to the present.

The starting point is the appearance of graphic novels such as The Dark Knight Returns (Frank Miller, 1986), Arkham Asylum: Serious House on Serious Earth (Grant Morrison, 1989), and The Killing Joke (Alan Moore, 1988), which re-envisioned Batman as a darker figure and laid the groundwork for the figure’s medial self-awareness.

In their wake, a key scene in Tim Burton’s 1989 film stages a parergonal reversal of medial spaces during Joker’s televised address to Batman and the people of Gotham. The medium takes on an unexpected materiality as the Joker shoves the mayor’s image off the screen, and a crucial reversal is visualized as a shot of several contiguous studio monitors gives way to the various screens united in Batman’s multimedia console. It is here, with a sudden freeze frame interaction, that Batman enacts a further parergonal reversal: while the film’s editing leads us to believe that Bruce Wayne, like all the citizens of Gotham, is viewing the Joker’s address live, he pauses the recording, in effect pausing the continuity of the film itself. And with this seemingly insignificant difference it introduces between live and recorded images, Batman’s pausing of the image announces, in effect, an entry into the interactive space of post-cinematic media. This is the first step towards the reconceptualization of images and visual media as purely processual, computational, and no longer tied to perception as its objects.

Jump ahead twenty years. Computational technologies are implemented more broadly in the actual production of visual media, for example in post-cinematic blockbusters like Christopher Nolan’s Batman trilogy. Nolan’s second film, The Dark Knight, can be seen as a serial continuation not only of Batman Begins, the first film in Nolan’s trilogy, but also an updating of Burton’s early exploration of Batman’s ergodic mediality.

Most centrally, Nolan’s film updates Batman’s console and places it in the middle of the caped crusader’s pursuits to restore order to Gotham. The film spends a considerable amount of time foregrounding this computational wonder machine, which alternates reversibly with the film itself and serves to foreground its CGI-based spectacles.

Within the frame of the narrative, a new range of computational powers is demonstrated, including biometric facial recognition and computational forensics. Early in the film, Bruce Wayne’s tech guy Lucius Fox demonstrates to him a new technology, utilizing a cell phone to emit an inaudibly high frequency capable of mapping a remote location by means of digitally enhanced sonar. This sets the stage for the film’s climax, when the sonar program is spread, virus-like, to the cell phones of all of Gotham’s inhabitants. Through this network, which feeds into Bruce Wayne’s central console, now equipped with a giant wall of display devices, Batman is able to “see” the whole city.

This is a disembodied or nonlocalized 3D computer-graphics vision generated through a distributed, nonhuman sensory form that substitutes computational process for perceptual object. Seeing through the eyes of a machinic network, Batman is able to find the bad guys just in time for the final showdown, but at a decisive moment Batman’s “vision” machine crashes.

The event is presented to us in first-person perspective, crucially drawing attention to the mediation of our own vision through computational processes. Here the parergonal reversibility between diegetic and medial levels is thoroughly parergodic, as we are made witness to an event that challenges the perceptual frames delineating the narrative and our ability to engage or disengage with the medium.

But the scene anticipates an even more intense experience of parergodic involvement in the video game Arkham Asylum. Here, a specifically parergonal exploration of spatialized boundaries between sanity and insanity that goes back to the graphic novel of the same title is translated into a narrative that weaves back and forth between “reality” and Scarecrow-induced hallucinations. The player, who has to act in order to stay alive, can never be sure when one of these hallucinatory states has begun, and he or she therefore gets drawn into such illusions until an abrupt awakening takes place in the wake of a victory (in a boss battle) or its deferral. Even more poignantly, though, there is a total break with all narrative, perceptual, and actional involvement at one point late in the game, when the images on the screen freeze and display digital artifacts and the soundtrack begins to skip like a scratched CD.

Suddenly, the screen goes black and the game literally reboots – at least, I could swear that my PlayStation restarted at this moment, while a feeling of panic gripped me. When the game restarts, we see images reminiscent of the game’s opening scenes – thus compounding a sense of fear that either my disk or my machine is broken, and that all my progress in the game, by this time some 10 or 20 hours, is lost and will have to be repeated from the beginning. But this time things are backwards: the Joker’s in the driver’s seat, escorting Batman into Arkham Asylum. The cutscene gives way to an interactive sequence where the player controls the Joker, thus instituting a weird sort of actional identification with the villain, who then turns and points a gun directly at the player, whose vision is suddenly realigned with the perspective of Batman. There’s no chance to avoid death, and we see this “Mission failed” screen with the tip, “Use the middle stick to avoid Joker’s gun fire.” Only, there is no middle stick on the PlayStation or Xbox controller. This whole sequence therefore emphasizes the point of interface as a reversible margin where computational or ergodic media converge as both the thematic/actional “content” and the material platform for play.

And the quasi-glitch and simulated crash of the game channel this attention to reveal the significant work involved in ergodic play—the very real panic and extradiegetic fears activated here highlight the cognitive and physical labor invested by the player, the precariousness of the digital platform for the storage or accumulation of such work, over which we have little individual control, though our activities are sure to generate profit for the corporations holding ownership of intellectual properties (like Batman), of proprietary software and hardware (like the console we’re operating), or the algorithms that will mine our activities for surplus value. This, I suggest, is parergodic culture.