The impetus for coming to this term was not a love of terminology, but the author's need as an artist to situate a set of experiments in creating "instrumental texts" and "textual instruments" within an appropriate context. While it doesn't make sense to discuss all of these experiments as games, what distinguishes them from other electronic texts is their playability — both that they can be usefully considered as playable, and their particular structures of play. This essay discusses, particularly, two "textual instruments" recently constructed by the author in collaboration with David Durand, Brion Moss, and Elaine Froehlich. While both of these instruments operate according to the logic of n-grams (as first used in textual play by Claude Shannon), one instrument is designed to play with known local texts while the other is designed to employ the contents of network RSS feeds and web pages. One composition for each of these instruments — Regime Change and News Reader, respectively — is considered.
I'm writing this in the first person. For clarity's sake. Because, while I am writing about terminology, the terms I discuss are simply those that I use to think through my work as an artist and its relationships with the work of others. If employed for different purposes, as we say online, "your mileage may vary."
The first term I need to consider is "game", but from an oblique direction. Because I need to consider some things that are "not games."
Figure 1: Hypertext fiction is not a game. A Storyspace map view of Bill Bly's We Descend [1].
"This is not a game" was a slogan, at one time, embraced by a group of hypertext writers and theorists. While it is often traced back to a 1988 hypermedia writing workshop run by Rob Swigart, the slogan's most visible proponent was John McDaid[2]. McDaid may be best known as the author of the "artifactual" fiction Uncle Buddy's Phantom Funhouse (published by Eastgate in 1993). Other proponents of the slogan included high-profile members of the hypertext community such as Stuart Moulthrop and Michael Joyce.[3]
Swigart has written for computer games and published traditional novels — at the time of the workshop he'd recently published the "computer novel" Portal through game developer Activision.[4] And, in fact, we can see the proto-slogan in use in discussions of Portal from before the convening of the workshop:
"Emerging from the mists of the 'Vaporware' list in PC Letter, Portal has at last been published by Activision. Having thus established that Portal is not vapor, its creator, novelist Rob Swigart, has some further observations about what Portal is and is not.
'It's not a game,' says Swigart. Nor, apparently, is it interactive fiction as we have come to know it. 'There is no parsing language in Portal,' he adds, 'no puzzles to solve.'
Then what is it?
'It's a computer novel.'
And that is...?
'A novel that can be told only through the medium of the computer'."[5]
McDaid writes the following of the workshop group's enthusiasm for "this is not a game" as a phrase: "We thought this so important that we put it on the t-shirt, in real big letters." Why was it so important? In part, McDaid tells us, it was practical. There was no way to compete with games, especially graphically (figure 1), so it was better to stay off that turf. But he also writes of what he calls a "larger, fictional issue." An issue connected to the structure of game play:
"The payoff for 'correct' play [is] usually to win; to play 'incorrectly' is to lose. This is very much at odds with what one might loosely call goals of fiction: exploration, insight, and the renewal of the perceived world through alterneity."
But McDaid then goes on to say, in the next sentence, "though it is true that in my own fiction, Uncle Buddy's Phantom Funhouse, there is in fact a puzzle." Faced with this, I'm a bit puzzled myself. Certainly I wouldn't want to elide the very real structural differences between fictions like McDaid's and those for which Infocom is best known. At the same time, however, there are also significant differences between McDaid's and Swigart's — not the least that one has "no puzzles to solve" while in the other "there is in fact a puzzle." Why, in these circumstances, would these writers have chosen such a primary focus on the term "game"?
When I interviewed Stuart Moulthrop, I asked him about this focus, about the hypertext community's version of "this is not a game." He pointed out that the implied comparison between the works on each side of the phrase wasn't neutral:
"There was an element of rank professional jealousy, for sure. [Infocom] had a market, after all. We were stuck in the garage. In retrospect our allergy to games looks incredibly foolish, both because Infocom's market experience didn't end all that happily, and more important because there was so much good work coming out of [the interactive fiction] community, and still is.
I suppose what really changed my mind on this was [the] reception hypertext has sometimes got from the literary community: 'How dare you? You have no place at this club.' The notion that we could have gotten similarly clubby, trying to exclude someone else's work in new media, now seems repulsive."
Here we see "this is not a game" functioning as a distinction between mere text games and work worthy of consideration by the literary community. Between low and high culture. Between trivial play and serious writing.
Interestingly, those on the other end of this distinction seem to have chosen a different approach. As it happens, the piece quoted above discussing Swigart's Portal was published in Infocom's house publication, The Status Line (previously The New Zork Times). It concludes: "Can interactive storytelling work without challenging puzzles or conflict resolution? Portal proves it can." This is not an endorsement of a competitor's product (Infocom had recently been purchased by Activision) but, unless motivated entirely by command of their new corporate owners, it is a somewhat surprising endorsement of an artistic project that the "other side" came to see as sharply incompatible with Infocom's.
Personally, I'm interested in the ongoing work of both of these communities, and I'm looking for terminology that can help me see the common ground that the authors of The Status Line recognized.
"This is not a game" has also served as material for an influential game, and from there become a slogan for the players and developers of a certain group of games.
The promotional game for the movie A.I. had no official name, but here I'll use its nickname: "The Beast"[6]. Just as the game had no official name, it also had no marketing (in fact, was unavailable for purchase), and no official beginning. Or, to put it another way, it began when and how people began to play it. For many it began with the second A.I. trailer, in which "Jeanine Salla" is credited as "Sentient machine therapist"[7]. Players' web searches for these terms revealed the beginnings of a trail that threaded through texts, images, and movies across the internet — as well as phone calls, faxes, US Postal Service deliveries, bathroom walls, and live events.
The game was a huge success — not only in the estimation of its players, but also in the surrounding media attention (which helped generate interest in A.I.). As the hype reached its height, in May 2001, a television commercial for A.I. was released that contained the words "THIS IS NOT / A GAME" (figure 2).
Figure 2: Stills from A.I. television commercial. "THIS IS NOT" and "A GAME" appear in red near the center of the first and second still, respectively.
As Jane McGonigal writes:
"This message has since become the mantra for both players and developers of immersive entertainment. To 'TING' a game now means to explicitly deny and purposefully obscure its nature as a game, a task that has become increasingly difficult as immersive players grow more savvy about TING techniques."[8]
Here "this is not a game" has a different feel to it. Still a slogan of sorts, but a game's slogan. And as McGonigal points out, TING is now a game design technique — one that helps support players in their performance of belief in the game's reality, a feature identified as key to enjoyment of the "alternate reality" gaming genre that The Beast helped launch. This denial of the game's apartness from reality may seem to refuse the "magic circle" that has been part of our discussion of games since Johan Huizinga's Homo Ludens. But rather than an erasure or breakage of the circle, it's probably more of a smearing or smudging — a deliberate extension of the border between a game and the rest of life, in order to create space for performance and play.
"This is not a game" could also be seen as a more formal characterization of The Beast. After all, it had no rules, no points, and only ill-defined outcomes toward which the players could work (locating what might be a puzzle, trying to solve it, hoping to find larger patterns leading toward the solution of the emerging murder mystery). But more on this sort of consideration is to come.
Some people are under the impression that The Sims (figure 3) is the best-selling PC game of all time[9]. In fact, as of this writing the publisher of The Sims, Electronic Arts, on its website leaves all qualifiers aside to call The Sims "The #1 best selling game of all time." But others would say that, while The Sims may have sold well, it is not a game. Rather, they say, The Sims is a "toy" or "simulation."
Figure 3: The Sims, in its original form, doesn't have a quantifiable outcome, or even clearly-defined goals for players to work toward. Is the best-selling PC game of all time also "not a game"?
One source of such arguments is formal game definitions. For example, Rules of Play, a game design text from MIT Press, defines games as "a system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome"[10]. The focus on quantifiable outcomes (which The Sims may not sufficiently possess) makes non-games, or borderline cases, of many experiences that we commonly call games, including simulation games and role-playing games. The closest these experiences get to meeting the definition is when considered in terms of interim quantifiable goals the players set for themselves. As Salen and Zimmerman put it, "As with other open-ended game-like experiences such as Sim City, RPGs have emergent quantifiable goals but usually no single overriding outcome"[11]. And these authors are far from alone in proposing formal game definitions that systems such as The Sims fail to meet.
But more "popular" publications also question the status of The Sims as a game. For example, several game publications' reviews of later installments (console versions of The Sims, or The Sims 2) note this attitude toward the first installment, for example:
"With the first game, people complained that The Sims was more like a toy than a game."[12]
"In the PC versions of the series, gameplay was typically thought of as a toy — something for players to pick up and enjoy for however long they want with no clear end in sight."[13]
Sometimes the difference is split:
"The original Sims was as much a toy as it was a game."[14]
Even the Wikipedia entry on The Sims also (as of this writing) notes that "It has been described as more like a toy than a game"[15].
Taking a different tack, some have sought to more clearly describe how The Sims deviates from the usual definitions of games and then describe this as an alteration of their game model (rather than a lack). Jesper Juul, for example, writes:
"Open-ended simulation games such as The Sims change the classic game model by removing the goals, or more specifically, by not describing some possible outcomes as better than others."[16]
This could result in us changing our game definitions or, as Juul does, classifying The Sims as a borderline game.
I don't mean, by this, to seem to be arguing against formal game definitions (or popular perceptions). It may well be that The Sims is not a game. At this point I'm simply intending to add "doesn't meet my (in)formal definition" to the senses in which people have meant "this is not a game."
Toys and simulations aren't the only types of entertainment software that "aren't games." In their 2002 SIGGRAPH presentation Michael Mateas and Andrew Stern put up a slide that read, in part, "It's a story, not a game"[17]. The two were referring to their jointly-developed project, Façade (figure 4), which provides a first-person dramatic experience for the interactor. There are no puzzles, no points to score, and no quantifiable outcomes. Instead, the interactor plays a character, interacts with other characters through language and movement, and has an experience shaped by Façade's software "drama manager." The experience, of a couple breaking up, has caused Façade to be described as "an interactive Who's Afraid of Virginia Woolf."
Figure 4: Façade's main characters, Grace and Trip, hold painful secrets and witty quips. Exploring the dramatic experiences possible with them, rather than achieving a higher score, is the motivation for replaying.
Mateas and Stern's project is part of a genre of entertainment/art software that has been called "interactive drama" or "cyberdrama" — popular knowledge of which has been spread by books such as Janet Murray's Hamlet on the Holodeck[18]. While interactive dramatic experiences may fail to meet many of the criteria of formal game definitions, this isn't what Mateas and Stern were arguing in their SIGGRAPH slides. Rather, they were seeking to draw attention to the different structures of play, and goals of play, in this genre. Here, there is no winning or losing, and the point of interaction is not to accomplish a game goal. The interactor certainly forms opportunistic short-term goals, but usually in terms of the dramatic situation. In this case, to be "not a game" is to be a different kind of playable digital media, calling for a different type of engagement, than usually comes to mind when the term "game" is used.
The assertion that interactive dramas are not games is relatively well accepted among game scholars. But it is not universally accepted, either within or outside such circles. And it seems that one place it is less accepted is within the game development community itself. Façade was chosen as a finalist for the 2004 Independent Game Festival awards, which must on some level be seen as acceptance of its status as a game by the jurors of that competition. Further, the IGF is held in conjunction with the yearly Game Developer's Conference — and, therefore, in 2004 Façade was on display in the main exhibition hall for the length of the U.S.'s primary gathering of game developers. I spent time at Façade's kiosk regularly during the conference and observed the people gathered around, watching others interact and waiting for their turn. I didn't hear a single one say that it was "not a game."
And neither Stern nor Mateas would have uttered the phrase then, either. By 2004 Façade's development had progressed further, and their presentation of the relationship between Façade and the term "game" had evolved considerably. While "it's a story, not a game" had helped shock some out of the "how do I win?" mindset, there is much in Façade that is usefully considered with the term "game." As Stern described it to me in an email:
"In fact, one of the underlying interaction mechanics of the first half of Façade we call the 'affinity game', where Grace and Trip interpret everything you do as a zero-sum taking sides 'game'; the second half of Façade we call the 'therapy game', where the player is (purposefully or not) potentially increasing each character's degree of self-realization about their own problems."[19]
Stern and Mateas here are making an interesting maneuver, introducing "game" in the interpersonal sense (as in Games People Play). As Stern explains in a post on the collaborative blog Grand Text Auto titled "Head Games," this isn't a way of trying to shift the ground away from discussion of computer games, but to imagine a new area of the computer game field which focuses on interaction with richly realized characters. Stern writes that, in such games:
"The gameplay will literally need to be about the characters themselves. The 'state space' the player manipulates — the variables you affect, the values you change — need to be the feelings, emotions and thoughts of the characters, not just external counters, scores, levels and objects.
"Rather than just firing a gun to cause an enemy's health to decrease, or a crate to explode or a door to open, you'll fire off discourse acts such as praise, criticism, expressions of feeling, requests and ideas; the other characters' attitudes will immediately change, emotions will get generated, and new actions will become motivated.
"What kind of game would that be? It could be the game of persuasion, or negotiation, seduction, or communication, for example. The kinds of games we play with each other all the time, really."[20]
Just as Façade attempts to integrate such games into an overall dramatic structure, more traditional game developers could integrate them with mainstream game genres involving human characters — from role-playing games such as Knights of the Old Republic to simulation games such as The Sims. Of course, for this to be successful, more research of the sort being undertaken by Mateas and Stern will be necessary.
All of the above uses of "this is not a game" are potentially fruitful. But the distinctions they make are not the ones I'm after. I'm not looking to separate high from low, well-demarcated from immersive, or the formally defined from border cases. I'm looking for a way to discuss all these examples together, in a manner that highlights a set of features that are of interest to me, and without throwing so broad a net that the weight of what I'm trying to pull in capsizes my vessel.
Taking this point of view, I have a series of thoughts: It may be that none of the examples in the preceding section are games, or some of them may be — and that's fine. Some of what I create as an artist may be games, or not — and that's fine, too. But thinking about how these may or may not be games has led me to identify something that is interesting about all the preceding examples, and hopefully also about some of what I'm creating as an artist — how they are played.
And this is what has led me to talk about "playable media." For me, this phrase shifts my thinking from a question I've found only temporarily useful ("Is this a game?") to one I have found rewards sustained attention ("How is this played?"). "Playable media" also encompasses a body of work that I want to consider — including the examples above, as well as many other products of the commercial game industry, and also the body of what might be called "playable art." By "playable art" I primarily mean projects from the digital art community such as Camille Utterback and Romy Achituv's Text Rain (figure 5), which invite and structure play.
Figure 5: Text Rain shows interactors a video image of themselves in an alternate reality. The letters of lines of poetry fall from above, coming to rest on anything darker than the background — inviting creative play with this language made physical.
Of course, I could also talk about all of these things by using an already-popular umbrella term such as "interactive." But these terms are often overly broad even for my purposes, and the impetus that allows them to attain their breadth may shape their meanings in a way that isn't helpful. For example, while it's easy to imagine an argument about whether a love letter is more "interactive" than a computer game, it's unlikely that we would spend long arguing which is more fruitfully considered in terms of its playability. And while playability is a useful way to look at agreed-upon games (such as football) it's also appropriate when considering certain types of related play (such as hackey-sack) without encompassing too much (it leaves aside sports commentators, the symbolism of team logos and names, the economy of sporting goods, etc.).
And a focus on the playable also attracts me for another reason — because we play more than games and "not a" games. We also play instruments, and compositions. And it is at the juncture of these senses of play — that for games, and that for music — that a thought-provoking discussion about playable texts has been taking place in the electronic literature community.
There are two types of playable texts that interest me here. The first type, "instrumental texts," has (as noted above) been the focus of some discussion in the electronic literature community. The second, "textual instruments," began as a personal thought experiment — a perhaps contrarian inversion of some of the assumptions of the first category. But then this thought experiment evolved into my first set of collaborative projects, perhaps the first of many, with textual instruments.
Let's begin with the first type. In electronic literature circles — those in which experimental writing for digital media is a common topic — the last few years have seen increasing discussion of the concept I'm calling "instrumental texts." (Within this phrase I'm subsuming a discussion, around texts with instrumental qualities, that has used a variety of loose terminology.) These are texts meant to be played. As John Cayley put it in an interview with Brian Kim Stefans:
"My point is that we are currently writers trying to build relatively simple textual instruments that are intuitive and, hopefully, both affective and significant when they are played. I mean played as musical instruments or sequencers or mixers are played. This is ergodic indeed, but still distinguishable from (hard) work or from the type of play in games which is rewarded by winning, by other forms of 'success' or simply by 'playability'."[21]
With the term "ergodic" Cayley is referencing the work of Espen Aarseth, whose Cybertext: Perspectives on Ergodic Literature[22] is a touchstone in the electronic literature community. Aarseth defines the term by stating, "In ergodic literature, nontrivial effort is required to allow the reader to traverse the text." More recently Aarseth has become one of the leading figures in the emerging field of game studies, helping found the field's first journal as well as the Center for Computer Games Research at the IT University of Copenhagen. In referencing Aarseth's work, Cayley brings to the fore a focus on play as "nontrivial effort" in music, games, and instrumental texts.
In my interview with Stuart Moulthrop cited above, I had the opportunity to ask him about instrumental texts. He, too, discussed some of the potential challenges of the reader/player's nontrivial engagement, but also some of what authors of instrumental texts might learn from the designs of folk instruments:
"Maybe some instruments will be hard to play. They may require practice. Or not. As a teacher once said to me about the guitar, 'After five or ten minutes you'll make sounds that are almost musical. That's what the frets are for.' And that's a great virtue of folk instruments. They do allow you to get in touch with a productive vocabulary very quickly. I think a good instrument would do that. It would stimulate engagement. It should make people want to get in there and interact, and to repeat the experience."[23]
But for Moulthrop, perhaps unlike Cayley, musical and literary figures are not the only ones brought together in this discussion. For Moulthrop musical figures are a vocabulary that can help one imagine projects that occupy a space between two other types of work at play in discussions of instrumental texts. As Moulthrop puts it:
"What I'm particularly taken with is the notion of a middle space between literary texts and ludic texts — between interactive fiction, or hypertext fiction, and games. You have, with instruments, a text with behavior and temporal dimensions that in some ways maps onto the temporal experience and interactive possibilities in game design."
In focusing on games, Moulthrop has likely chosen the stronger of the two comparisons. For the examples usually given of instrumental texts, unlike musical instruments, only "play one tune." Their structures of play and material for play are designed and delivered together, much as those of games. But this is not to say that the musical figure is unimportant. Rather, I think it helps indicate the sort of engagement that authors of instrumental texts hope audiences will have with their work — again, as with Façade and The Sims, something that is not about winning or losing, or perhaps about quantifiable outcomes of any sort. But, unlike Façade and The Sims, this engagement is a physical discipline, sensitive to differences in movement and able to be learned at a muscular level.
Cayley's interview statement about instrumental texts actually came in the context of an answer to a question about his piece riverIsland (figure 6). The occasion of my interview with Moulthrop was the release of his piece Pax (figure 7). Looking at these may help ground our discussion, as well as help us understand what other work might be brought into this category.
John Cayley's work often employs a technique he calls "transliteral morphing." This is a letter-by-letter morphing that transitions from one text to another, much as graphical morphing moves points in space so as to transition from one image to another. In transliteral morphing the in-between letters are determined by movement along a loop on which Cayley has arranged Roman characters according to their sounds, as he explains:
"If texts are laid out in a regular grid, as a table of letters, one table for the source and one table for the target, to morph transliterally from one text (one table of letters) to another, is to work out, letter-by-letter, how the source letters will become the target ones. Assume your alphabet (including 'space' and apostrophe, 28 letters in all) is arranged in a special loop where letters considered to be similar in sound are clustered together. The aim is to work out the shortest distance round the loop (clockwise or anti-clockwise) from each source to each target."[24]
Once the movement for each letter is worked out, the text then moves through fourteen steps (the largest number that might be necessary for any one letter — movement to the opposite side of the 28-character loop). Some letters go through many more transitions than others. Changes are "reluctant" at the beginning of the process and then "anxious" for completion at the end — so that both the early and final stages are close to readable texts.
A number of Cayley's pieces, such as his well-known windsound[25], employ transliteral morphing in a manner that is performative on the part of the program. Texts morph into other texts under the gaze of the reader/audience, using the computational capabilities of the computer on which they are displayed. And yet these morphs could be, like most of the graphical morphs we see, pre-rendered and displayed as moving images (without any computation at the time of reading). The only visible loss would be the small changes in timing from reading to reading on the same computer, and the occasionally larger changes when moving from computer to computer.
Cayley's riverIsland[26], on the other hand, is not only performative on the part of the system, but also performative in a manner controlled in part by the reader. One of the types of performance made available to the reader is relatively straightforward: riverIsland is composed of two loops of poems, one horizontal and one vertical, and the reader can use on-screen arrows in order to trigger movement along these loops. When the reader indicates that a move should be made from one poem to another, the appropriate transliteral morph is performed by the computer.
There is another type of reader performance in riverIsland, however, that feels quite different to me. And I believe that this is part of what Cayley was getting at in his talk of instruments during his interview with Stefans. In this type of performance, the reader can click and drag on the screen's vertical and horizontal Quicktime movies. The vertical movie is an "object" movie that graphically transitions between images of paths through the woods. The horizontal movie is a panorama of a riverside scene. A reader experienced with riverIsland can use these movies to navigate to any point within the work's two loops. A transliteral morph is then performed between the text that was being displayed before the navigation process began (which might, itself, be an in-process morph) and the destination selected by the manipulation of a movie. This creates an experience for which pre-rendered morphs could not effectively substitute — like Cayley's figure of the sequencer, it harnesses real-time computational processes to create a performance based on high-level user direction that requires knowledge of its materials and control space.
Figure 6: riverIsland enables traditional, step navigation through its textual morphs via the arrows on the lower-middle right — or, for those with knowledge and practice, more free-form selection of destination texts through the horizontal panorama and/or vertical object movie.
The instrumental text of Stuart Moulthrop's that I will consider here, Pax[27], presents an experience of reading and performance that differs from riverIsland. Its differences in some ways map onto two of the different musical instruments Cayley and Moulthrop chose for their examples when discussing instrumental texts — while Cayley mentioned the sequencer, Moulthrop mentioned the guitar.
A sequencer might play itself for some time after being given instructions, but a guitar demands interaction for each note sounded. Similarly, Pax is structured for near-continual interaction. The larger area of the piece, on the left, shows characters floating up (in the first half of the piece's duration) or falling down (in the second). Unless the reader interacts with these characters, almost no text appears. Readers interact by "catching" floating characters with the mouse pointer. Characters can be released by moving the mouse away, or clicked (either by active clicking, or by holding them caught for 20 seconds). Clicking elicits text from that character, which appears in the area on the right (this becomes a scrolling text area once there is enough text to scroll). The fourteen characters float by in different orders, but those recently clicked tend to reappear, making it possible to consistently evoke text from two or three characters as the piece's time passes. Each reading lasts from noon to midnight (the characters' time) and is divided into six thematic movements: "Shaken Out of Time," "American Flyers," "Home Land," "Evil Ones," "Falling," and "Total Information." The text elicited from a character is determined, in part, by the number of times the character has been caught and clicked, as well as the current movement of the piece. The character texts evoke two situations: being caught in some version of a terminal at the Dallas airport (shut down for security reasons in an even-more-irrational "war on terror" than that which now grips us) and being caught in the space and structure of Pax itself (naked, floating, caught and prodded by the interactor).
While it would be impossible to manipulate the Quicktime movies of riverIsland toward particular effects without relatively strong knowledge of the piece, Pax provides obvious places to click and quickly-understood effects even for the first time reader. However, because of its random elements and the strong impact of time's passage, it would be more difficult to exactly reproduce the same reading (after learning to play) than with riverIsland. To put it in terms of the musical analogy, Pax may provide frets, but for an instrument that adjusts its tunings over the course of each playing.
And, this, again, points to the strength of computer gaming as a figure for understanding instrumental texts. In the gaming context there is nothing surprising about behavior that changes over the course of time. There is also nothing surprising about the skills of physical manipulation and memorization that would be required to elicit particular readings from riverIsland and Pax. And the fact that these "instruments" come packaged with only one composition, from which they cannot be easily decoupled, also makes sense in the context of computer games. And yet they are clearly not games in the manner that play is approached. Perhaps what the musical analogy helps with most is the fact that these projects seek a lyric engagement — not easily formulated in terms of contest or quantifiable outcome.
Figure 7: Pax produces texts when the reader catches and clicks on characters that float by — and is otherwise silent. Rather than a narrative "told" to the reader, or one "played through" as in (for example) the levels of a narrative first-person shooter, Pax is an exploration of character and situation.
With a better sense of what we mean by "instrumental texts" it may now be possible to adopt into the category a number of computational textual projects not described by their authors in such terms. In fact, I'd like to propose a perhaps radical move — adopting as an instrumental text a project that uses a method of interaction not even designed for text, but repurposed through use of an existing game engine. I believe the adoption is appropriate once we look at it, and this foregrounds the game-like structures of interaction for instrumental texts, as well as the different — more musical or performative — position of engagement with these structures.
The piece I propose adopting for these purposes is New Word Order by Sandy Baldwin (as reported in Funkhouser[28]. In this, the second part of Baldwin's Black Mesa project, poetry is presented mapped onto objects in a simple Half-Life level. The poetry is that of Billy Collins, the first U.S. poet laureate (2001-2003) appointed during the presidency of George W. Bush. Placing Collins's poetry within Half-Life subjects it to destruction and reconfiguration with an arsenal ranging from automatic weaponry to the famous crowbar (figure 8).
Figure 8: New Word Order invites the reader to reduce and reconfigure poetry using the interaction structures of the first-person shooter.
New Word Order takes an interaction structure invented for competitive play with quantifiable outcomes — for gameplay — and repurposes it as play that recontextualizes and explores the potential of poetic language.
When interviewing Moulthrop, it struck me that one of my in-process collaborative projects, Screen, might also be usefully considered as an instrumental text. Screen combines familiar game mechanics with virtual reality technology to create an experience of bodily interaction with text. At the same time, the language of the text, together with the uncanny experience of touching words, creates an experience that doesn't settle easily into the usual ways of thinking about gameplay or VR.
Screen is a collaboration with Andrew McClain, Shawn Greenlee, Robert Coover, Josh Carroll, and Sascha Becker that was created in the Brown University immersive virtual reality chamber (Cave), as part of a research project in spatial hypertext writing[29]. Brown's VR chamber is similar to the University of Illinois's CAVE — a virtual environment that shows three-dimension images while allowing users to continue to see their own bodies, and that does not require users to wear encumbering equipment (unlike head-mounted displays, which are essentially blindfolds with televisions inside)[30]. Brown's Cave is an eight foot cube, missing its top and one side, and its walls and floor are screens. A Projector is pointed at each screen, alternately projecting streams of images meant for the user's left and right eyes. The user wears shutter glasses that alternately occlude the left and right eyes, in synchronization with the projectors. The result is stereo VR — 3D vision of computer-generated imagery — combined with the physical presence of the people and objects in the Cave.
The initial experience of Screen can be disorienting for those familiar with VR. Rather than make the walls "disappear," we project flat images onto the same plane as the walls, reinforcing their presence. And the images we project are not of colorful shapes, but of white text on a black background. This text at first appears in an introduction that fades in and out on the walls — and then forms three traditional paragraphs, each nearly filling one of the walls. Each of these paragraphs is a character's moment of memory that gives rise to the virtual experience of touch. Each wall appears, and then is read aloud. After the last has been read there is a pause, and then a word peels from one of the walls, is spoken aloud, and flies toward the reader. If the reader does nothing, the word circles near her. Soon another word peels, and then another, at an increasing pace, flocking around the reader. The reader can intervene in this process by batting at words with her hand. When a word is hit a sound is heard, and the word flies back toward a wall, perhaps breaking apart in the process. If a hit word is the only word off the walls it will return to the space it left empty. However, if more than one word is off the walls then a hit word may return to a different space.
Figure 9: Words collapsing in Screen.
Once the number of words off the walls passes a certain threshold — something which, with the increasing pace of peeling, only very active engagement can long delay — all the remaining words come free of the walls, swirl around the reader, and then collapse into the center of the Cave (figure 9). A final "closing" text is then heard. In addition to creating a new form of bodily interaction with text, Screen creates three reading experiences — beginning with the familiar, stable, page-like text on the walls; followed by the word-by-word reading of peeling and hitting (where attention is focused); and with, simultaneously, more peripheral awareness of the arrangements of flocking words and the new (often neologistic) text being assembled on the walls. Screen was first presented in 2003 as part of the Boston Cyberarts Festival, and in 2004 it was included in "Alt+Ctrl: A festival of independent and alternative games" at University of California, Irvine.
Given its presentation at Alt+Ctrl, we might simply discuss Screen as a game, rather than with the more unusual term "instrumental text." And, in fact, the final moments of Screen feature a scattering of words (and parts of words) on the walls — which caused one young visitor to the Cave to ask, "Is that my score?" But while the play of Screen is reminiscent of classic games from Whack-a-Mole to Breakout, and some players may at moments be driven purely by the game-like goal of hitting words as quickly as possible, there is no contest or quantifiable outcome. Even approached purely physically, without any attention to the linguistic nature of the words being played, Screen is more like hackey-sack than soccer/football. And, in my observations, players don't approach Screen without attention to its words as words. Rather, interactors oscillate between reading and playing, with the objects of both coming faster and coming apart, until both experiences can no longer be sustained and the piece ends. As with riverIsland and Pax, reader/players can get better at Screen, though the fact that interactors do not control the ripping of words alters what is possible via virtuoso performance. Perhaps the most impressive performance of Screen I have seen is that of Michelle Higa, who both edited the video documentation of Screen and played the role of the interactor within it.[31] In order to videotape Screen we had to turn off the flickering alternation between left- and right-eye images. Higa had become adept enough at the experience of Screen that she was able to play it relatively successfully even without stereo cues.
I first discussed the idea of "textual instruments" in a short paper for Digital Arts and Culture 2003.[32] The idea grew from questions about the limits of instrumental texts. If instrumental texts are odd instruments in that they only play one tune, how might we imagine tools for textual performance designed to play a variety of compositions? What would it mean to have textual instruments that one might learn to play proficiently, for which one might write and perform a number of compositions, and that could eventually be made available to play the compositions of others?
In my DAC paper I described textual instruments as follows:
"A textual instrument is a tool for textual performance which may be used to play a variety of compositions. In this sense it is evocative of Thalia Field's figure of the 'language piano'— something that one learns to play, and which may produce a much wider variety of texts than is the case for those projects normally discussed as instrumental texts.
"However, a textual instrument need not be like a prepared piano. The direct selection of text, rather than the manipulation of a non-linguistic device, can be its interface. And the relationship between a textual instrument's interface affordances and the possible textual outcomes need not be one-to-one at all levels (as it must be with a piano's keys, though they may be played in many combinations). Understanding at a gut level how a textual instrument's probability spaces function for a given composition is part of learning to play that piece.
"Compositions, here, consist of a body of text (and/or a means of acquiring text) and a set of 'tunings' for the instrument(s) used."
While I don't know of any projects, other than those I have been involved with, that are described by their creators in these terms, in the next section I'll adopt a potential example into the category. I'll then touch on a couple of related issues before discussing my first two collaborative projects in this area — the compositions Regime Change and News Reader, and the instruments for which they were composed.
Most who approach the arts as writers are quite attached to their own words, and this holds true among writers for digital media. In the digital field, Jim Andrews is one of the exceptions, having undertaken a number of interesting projects that involve him arranging a system for language to inhabit and then (rather than including only his own writing) inviting other writers to provide text. Arteroids[33] is one of these systems (figure 10).
Figure 10: Arteroids uses the interaction structures of a modified version of the arcade classic Asteroids, but replaces the images of a spaceship and rocks with images of text.
Arteroids is, on the interaction level, simply the repurposing of an existing game — and, in that way, quite similar to projects such as New Word Order. The major difference, in fact, is that while Baldwin's piece seems created as a context for Collins's work, Andrews appears to view Arteroids as an instrument for which many texts could be composed (and then used as materials for play). Andrews created a "Word for Weirdos" to allow others to compose for Arteroids and has included texts from others in presentations of the work — such as the texts by Christina McPhee and Helen Thorington included when Arteroids was shown in the "page_space" exhibition[34]. While on some level it may appear that learning to play Arteroids is no different than learning to play Asteroids, one can imagine the desire to create particular linguistic experiences changing this. With each textual composition as a different starting place, even after the Asteroids-derived parts of Arteroids have been mastered there might yet be much to learn about creating an evocative experience from the play with any one text.
And yet I don't enjoy Arteroids very much, even when it uses texts by writers whose work I have appreciated in other contexts, and even after gaining some experience with playing it. Considering why this is the case has led me to define further the work I want to pursue in textual instruments.
Not all playable computational media is graphical. In fact, some of the most popular early computer games were entirely textual. Games like Adventure and Zork were even at times played on teletypes, with the interaction recorded on scrolling reams of paper, rather than on terminals with screens. An excellent tracing of the history of this textual interactive fiction, which is still being created today, can be found in Nick Montfort's Twisty Little Passages[35].
But when we think of playing with computers, we generally think of graphical experiences, those that follow in the tradition of Spacewar! (figure 11) rather than Adventure. Created on the PDP-1 at MIT in the early years of the 1960s, Spacewar! was the first modern video game. Two players each had a custom-made controller, which they used to control the flight of a virtual spacecraft on the PDP-1's CRT. The spacecraft were pulled toward the star at the center of the screen by simulated gravity, and could fire projectiles at one another. A spacecraft hit by the central star or a projectile would be damaged. These are still among the central logics of graphical gaming today — the ability to move graphical objects that on some level represent the player, the ability to fire projectiles, a simulation of some form of physics, and "collision detection" when one thing runs into another. These logics aren't only the basis for play in experiences such as Half-Life, but also (leaving aside projectiles) in pieces such as Text Rain.
Figure 11: Spacewar! was the first modern video game, combining logics of graphical play still in wide use today.
We're accustomed to seeing successful combinations of graphical logics and game rules repackaged repeatedly. Games such as Pac-Man and Tetris have had many authorized and unauthorized versions "skinned" with different surface graphics and different graphical arrangements, but with the essential logics of graphical movement and gameplay preserved. Such combinations, within a larger range of variation, are also the basis for our identifications of game genres such as "side-scrollers" and "first-person shooters."
I bring all this up in order to make a point about the instrumental texts discussed above — about how they are played. While each of these pieces contains a textual component, they are all played along graphical logics. For riverIsland play is primarily through the graphical/physical manipulation of the Quicktime movies; in Pax it is the collision detection as characters are caught and clicked; in New Word Order the movement of the first-person perspective and collision detection (in firing of weapons and use of the crowbar); and in Screen it is the movement of the interactor's body and the collision detection of hitting words. What these projects do, in each case, is package together logics of graphical play and methods of response with textual and graphical material.
What Arteroids does differently is take a set of logics of graphical play and methods of response and then open them to many different sets of textual material. This might be seen as the same as taking the formula of Pac-Man or Tetris and opening it up to many variations in graphical representation. But, for me, it doesn't feel the same. Somehow it feels arbitrary, no different than if the graphics in Pong or Spacewar! were opened to replacement by arbitrary text.
I've been thinking about what would feel less arbitrary. It seems to me that, if the same graphical logic can be skinned with many different surface graphics successfully, perhaps those seeking to create textual instruments will need to consider forms of play that proceed via linguistic or textual logics. Before the computer became part of everyday life, textual forms of play such as the crossword (or games of the Surrealists or Oulipo, or Madlibs, and so on) successfully accommodated many different texts by structuring play around the features specific to textuality. Perhaps the true challenge of creating textual instruments involves finding such structures that benefit from the computational environment.[36]
If such structures can be found I also suspect that the result will feel, at least to me, in some ways like a deeper engagement with text than is possible withprojects (like Screen) that proceed along graphical logics — projects that could still be played, though perhaps not as rewardingly, if their words were all converted into colorful boxes.
Of course, the next question is where one might begin to explore such textual or linguistic logics. Though perhaps this work has already begun, with those creating computer-based versions of crosswords, Surrealist games, Oulipian games, and so on. And Fields of Dream[37], by Nick Montfort and Rachel Stevens, goes further — bringing the basic fill-in-the-blank logic of Madlibs into a project specific to the networked computer.
But there is also another territory to explore. This is that of linguistic and textual logics previously employed in text processing and generation — in contexts ranging from the computer science subfield of natural language processing (NLP) to the artistic contexts of John Cage, William S. Burroughs, or Jackson Mac Low. Whether considered scientific or artistic, these methods have generally been operated in batch mode — either generating chunks of language or analyzing chunks of language. But there is nothing to prevent them from being run interactively, or to prevent the interaction with them from being structured as play.
In fact, a number of the logics used broadly in NLP have already been used in play, if not necessarily computational play. Claude Shannon, sometimes referred to as the "Newton of the Information Age," even introduced one of these logics by describing a textual exercise quite similar in tone to Oulipian play structures such as "N+7."
Shannon's nickname comes from the fact that he was one of the major figures in formulating the mathematics of communication — what he called "the fundamental problem of . . . reproducing at one point either exactly or approximately a message selected at another point"[38]. This quotation comes from a paper that was a milestone in the field, both when published on its own (as "A Mathematical Theory of Communication") and when repackaged as a book with an introduction by Warren Weaver (as The Mathematical Theory of Communication). While a milestone, Shannon's paper certainly doesn't have to be approached as a millstone. Rather, it can be approached playfully, an approach to the products of science and technology practiced by Shannon himself. (His famous projects of this sort include a mechanical rat for solving mazes, figure 12, as well as a machine that, when turned on, performs only one action: causing a mechanical hand to reach out from a box and turn the machine off.) One aspect of the paper amenable to a playful approach is Shannon's description of stochastic approximations of English.
Figure 12: Shannon with his mechanical mouse.
Shannon's discussion includes a number of sample messages. One is purely random. But the others have contents that are shaped by the frequencies of particular letters or words in the English language. This shaping is of two sorts. In one type of shaping, individual letters or words are selected in a manner weighted by their frequency in English. In the second type of shaping, letters or words are selected in a manner shaped by the frequency with which they appear in groupings of letters or words in English. So, for example, "E" is a more common letter than "U" in English. However, if there is a pair of letters that begins with "Q" it is much more likely that the complete pair will be "QU" than "QE." Taking the frequencies of pairs into account in this manner means paying attention to the frequencies of "digrams." Paying attention to sets of three is attention to "trigrams." And, more generally, paying attention to the frequencies of groups of symbols (rather than only individual symbols) is the use of "n-grams."
Shannon provides six sample messages. In the first, each of the 26 letters and the space appear with equal probability:
"XFOML RXKHRJFFJUJ ZLPWCFWKCYJ FFJEYVKCQSGHYD QPAAMKBZAACIBZLHJQD."
In the next, the symbols appear with frequencies weighted by how commonly they appear in English text (i.e., "E" is more likely than "W"):
"OCRO HLI RGWR NMIELWIS EU LL NBNESEBYA TH EEI ALHENHTTPA OOBTTVA NAH BRL."
In the third, symbols appear based on the frequencies with which sets of two of the symbols in English. That is to say, after one letter is recorded, the next is chosen in a manner weighted by how commonly different letters follow the just-recorded letter. The sample message created in this way is:
"ON IE ANTSOUTINYS ARE T INCTORE ST BE S DEAMY ACHIN D ILONASIVE TUCOOWE AT TEASONARE FUSO TIZIN ANDY TOBE SEACE CTISBE."
In the fourth, symbols appear based on the frequencies with which sets of three of the symbols appear in English. This, again, is called a "trigram" — with each choice of the next letter being weighted by the frequencies with which various letters follow the set of two just recorded. This sample message is:
"IN NO IST LAT WHEY CRATICT FROURE BIRS GROCID PONDENOME OF DEMONSTURES OF THE REPTAGIN IS REGOACTIONA OF CRE."
In the fifth, the unit is moved from letters to words. In this message, words appear in a manner weighted by their frequency in English, but without attention to the prior word:
"REPRESENTING AND SPEEDILY IS AN GOOD APT OR COME CAN DIFFERENT NATURAL HERE HE THE A IN CAME THE TO OF TO EXPERT GRAY COME TO FURNISHES THE LINE MESSAGE HAD BE THESE."
Finally, in the seventh sample message, words are chosen based on the frequency with which pairs of words appear in English. This, again, like the technique of choosing based on pairs of letters, is called a "digram" technique. The final message is:
"THE HEAD AND IN FRONTAL ATTACK ON AN ENGLISH WRITER THAT THE CHARACTER OF THIS POINT IS THEREFORE ANOTHER METHOD FOR THE LETTERS THAT THE TIME OF WHO EVER TOLD THE PROBLEM FOR AN UNEXPECTED."
These messages are interesting in part because of how they are generated. The first two were created using a book of random numbers, with the addition of a table of letter frequencies when creating the second. But the rest of the samples were constructed by using ordinary books (making the assumption that ordinary books appropriately reflect the frequencies of letters and words in English). Shannon explains the process as follows:
"To construct (3) for example, one opens a book at random and selects a letter at random on the page. This letter is recorded. The book is then opened to another page and one reads until this letter is encountered. The succeeding letter is then recorded. Turning to another page this second letter is searched for and the succeeding letter recorded, etc. A similar process was used for (4), (5) and (6). It would be interesting if further approximations could be constructed, but the labor involved becomes enormous at the next stage."
That is to say that the last sample message (which begins with a sequence that sounds surprisingly coherent) was created by opening a book to a random page, writing down a random word, opening the book again, reading until the just-recorded word was found, writing down the following word, opening the book again, reading until that second word is found, writing down the following word, and so on. This is a method that sounds remarkably like some of the kinds of textual play engaged in by experimental artists and writers. It certainly sounds like a technique that could be brought into an explicitly playful context.
The surprising coherence of the last sample is also of interest. It shows that processes of the sort being used by Shannon have potential as a form of logic that can be made operational with linguistic material. Of course, the sample is quite rough, in part because only one previous word is being taken into account at any time. What Shannon calls "further approximations" — for example, taking two, three, or more previous words into account — certainly gives more English-like results. Shannon pointed out correctly that, using the method of flipping through a book, the labor involved in creating such further approximations would be enormous. But the modern availability of computing power has made carrying out such calculations automatically a near-trivial task for reasonably-sized bodies of sample text. As Shannon also pointed out, the stochastic processes he described are commonly considered in terms of Markov models. And, interestingly, the first application of Markov models was also linguistic and literary — modeling letter sequences in Pushkin's poem "Eugene Onegin"[39]. But Shannon was the first to bring this mathematics to bear meaningfully on communication, and also the first to use it to perform text-generation play.
This model (whether called n-grams or Markov chains) is now widely used in natural language processing and generation, often in combination with other techniques. It has also been used in electronic literature, perhaps most extensively by John Cayley. At least seven of Cayley's works employ "collocational" word-level digram procedures, including Book Unbound[40] as discussed in Aarseth[41]. Last but not least, this approach has also been the primary basis of textual toys such as the DOS program Babble!, the emacs "Dissociated Press" command, Hugh Kenner and Joseph P. O'Rourke's Travesty, Andrew Plotkin's chan.c, and Brion Moss's prate — which have themselves at times been used in the generation processes for (non-playable) literature. However, as noted above, both the elit and toys based on n-grams have operated entirely in "batch mode." That is to say, the interactor requests a body of text, and then that text is produced — following which the text can be read and another text can be requested, but no interaction with the texts (or interaction during generation) is possible. Given this limited nature of play with n-gram texts, there is also limited context for play — usually a blank text buffer for the program to write text into. After talking with Moss (with whom I'd collaborated on The Impermanence Agent[42]) about these issues, we began to imagine possibilities for n-gram play that was less batch-oriented and took place within a textual context.
Moss and I approached Turbulence, an organization that supports digital media art, and they commissioned us to create two pieces. These pieces would be inspired by the idea of textual instruments and operate using the logic of n-grams. After a false start with different collaborators, Moss and I connected with document researcher David Durand (best known for his work in formulating a number of document markup standards, including XML) and designer Elaine Froehlich (principal of Active Surface Design). From there, the project's conceptualization and execution were a team effort, with the initial technical work happening on top of Moss's Java prate, and later development built on top of work done by David Durand in Tcl/Tk.
Two major design decisions were made early on. The first was that, rather than build an n-gram text into an empty text buffer, play would always begin within the context of a pre-created document and consist of progressive alteration of that document. This was motivated, in part, by the fact that, while the text produced by n-gram algorithms has microstructures that are recognizable from its source texts, the larger structures of n-gram texts tend to be very similar regardless of the starting material. Some have tried to address this by looking at larger structures in the source texts statistically, but unless the texts in question have been marked up by a human author or editor, this process involves a series of assumptions about the text (e.g., that a period marks that end of a sentence, as it does not in the case of "e.g.") that are both sometimes inaccurate and on some level aesthetically displeasing. These assumptions are displeasing because they depart from the purity of the simple n-gram algorithm, which in its basic form would work with starting texts in Japanese or Braille or musical notation as easily as English-language ones. However, there was also another motivation. In many n-gram texts, especially those based on short chains, part of the pleasure is based on play between coherence and incoherence — and we found something more interesting, and potentially more meaningful, in such borderline coherence occurring within the context of traditionally-created texts.
The second design decision was the identification of our basic method for making n-gram generation playably interactive, rather than oriented toward large batches. We decided that, in addition to the starting document (within which play takes place) we would have a body of text used for producing the alterations to the starting document. (We call this second body of text the "alteration text" or "alteration corpus.") When the starting document was displayed, certain words would be highlighted. We chose this as a convention familiar from hyperlinks on web pages, letting interactors know that a click will elicit a response. However, these words are not highlighted as the result of author-specified links. Rather, they are highlighted because a string of n-gram text (of a length specified by the piece's author) appears in both the starting document and the alteration text. We decided that such "bridges" between the two bodies of text would offer interactors the opportunity to open up the starting document and insert text generated from the alteration corpus. More than one generated text would be offered for possible insertion, allowing the interactor to choose one or none (this last leaving the text unaltered). The texts offered would, themselves, be generated from the alteration text through the use of n-gram techniques. The number of texts offered and the n-gram length used in their production would, again, be determined by the piece's author.
Once these decisions were made, we sketched, mocked up, and eventually tried to make operational a number of interaction designs. Some didn't give the kinds of results we'd hoped for, and others were too computationally expensive to work, but we eventually settled on one that — for us — is satisfying in terms of the feel of interaction and the shape of the attention to text it creates. We're still discussing some potential variations on the visual/spatial aspects of the interface, but for the first compositions (Regime Change and News Reader) a simple, web-style series of windows seemed both appropriate and easy to implement.
More information on the details of interaction will appear below, in the context of the discussions of these two compositions. But first it's worth noting what differentiates the two instruments we constructed. Our first instrument was the simpler of the two. It depended on both the starting document and alteration text existing on the local drive, with known file names. The second instrument adds a number of features, including the request, processing, and display of network RSS feeds and HTML files. As a result, the first instrument is better suited to compositions involving longer chain lengths (and therefore greater coherence) because it doesn't have to take time for network file requests or for processing the wide variety of network html files into text that the system can use. The second instrument is, of course, more suited to compositions involving dynamic network data sources — in fact, we regard it as a type of alternative browser.
While some of this discussion may have been a bit difficult to follow in the abstract, hopefully the following two examples will make things clearer.
Regime Change begins with a news article from April 2003, following the bombardment that began the U.S. invasion of Iraq[43]. George W. Bush cites "eyewitness" intelligence that Saddam Hussein was assassinated by targeted U.S. bombing, and clings to the contention that the Iraqi president was hiding "weapons of mass destruction. "Playing Regime Change brings forth texts generated from a document that records a different U.S. attitude toward presidential assassination and eyewitness intelligence — the report of the Warren Commission.
Figure 13a: Regime Change displaying its starting text.
Once the window with Regime Change's starting text is opened, words in that text, pair by pair, become highlighted. Clicking on words opens a new window (figure 13a). Interacting with new windows produces new texts that will take the place of the clicked words.
New windows contain texts that begin with the words clicked in the previous window. Each paragraph in the new window is an alternative text — beginning with the same words but potentially (though not necessarily) following many different paths from there. These texts are generated by connecting chains of words (3-grams and 4-grams) that may have appeared originally in very different parts of the source document.
A new window's texts, once displayed, also begin to have words highlighted within them. Clicking highlighted words will open another new window, containing generated texts that can take the place of the clicked words (figures 13b-c). Opening several generations of windows opens wider possibility spaces for the texts that will be created (and that will replace the clicked words in earlier-generation windows). Windows alternate between generation from the Warren Commission text and the original news story.
Figure 13a-c: Regime Change opening multiple layers of windows.
In any window with generated text, clicking a non-highlighted word is also a means of interaction. Such a click will close the window — and select a text. The selected text will run from the beginning of the clicked paragraph to the clicked word. That selected text will then take the place of the words clicked to open the window (figure 14a-b). This creates a kind of stretchtext — the pair of words clicked to open a window are replaced with the words selected in the open window (usually more than a pair).
After opening several layers of windows, part of play is keeping track of where each window came from — so that it can be collapsed by selecting a word that will make a pleasing segue at the point where it will join the text to which the player intends to connect it. (This may be more than one layer down.) Keeping track of context is made easier by the title bar of each generated window — which displays the two words that will be replaced by the generated text, followed by the two words that appear after them in the text clicked.
I find that, when I'm playing, this cycle of activities — reading, remembering context, selecting a place to click, reading again — consumes my entire attention. I've found it impossible to "give a reading" of Regime Change as I might with other writing projects. My most successful presentation so far, instead of a traditional reading, was a performance in which I played the text and Popahna Brandes read the results aloud.
Figure 14a-b: Word replacement in Regime Change.
Most of the interactions I've described are those that I would consider part of the instrument. I consider the composition to consist of the selection of texts, the basic settings made for the instrument (n-gram chain lengths, number of alternative texts generated, various selections of colors and fonts, and the ways that different windows open into different texts), as well as the ways that settings change over the course of interaction with the piece (including a "surprise" third text that becomes part of Regime Change's material after a certain number of third-level windows have been opened).
Of course, I'm aware that this particular set of terms is not the only reasonable way to understand Regime Change. Rather than a focus on concepts such as instrument, composition, and play, it would also be reasonable to view the piece in terms of something like Nick Montfort's "human-computer co-authorship"[44]. Such a focus would seek to make more explicit the moves made by system and interactor during the creation of the final textual output from a session with Regime Change. Employing Montfort's framework, we would say that the initial move is: Computer — G (the computer provides the initial text). Then: Interactor — I (the interactor provides "some instructions or intermediate text" by clicking). Followed by Computer — G (the computer generates texts based on where the click was placed). Finally, Interactor — I, which can lead to A by extension or Computer — G (the interactor clicks, which may result in alteration of text or the generation of further text). Then these final stages are repeated, indefinitely, until the concluding text is reached. This enumeration may seem an odd exercise, but it does help us formalize how the texts of Regime Change differ from those created in other situations of human-computer co-authorship. For example, while human participants certainly shape the text created during interaction with Regime Change, Montfort´s model makes it clear that interactors at no point generate text.
News Reader is software for reading the news, and for re-forming it[45]. It is a specialized browser — displaying a selected RSS feed, as well as the news stories to which the feed links. Unlike a normal browser, News Reader also downloads another set of texts in the background — and uses this material to open each page it displays to textual play. Through this play the concerns and language patterns of the hidden documents, as shaped by the movements and passages selected by the player, are introduced into the original news stories. News Reader provides a different way to encounter the daily news, making its patterns of repeated phrases into opportunities for disruption, and producing results that range from humorous to disturbing.
When News Reader launches it displays a window containing the current headlines from the Yahoo! News RSS feed (figure 15). Clicking the headline or preview text opens another News Reader window, displaying the story.
Figure 15: News Reader showing Yahoo! News RSS feed.
A link below the text preview copies the address of the story to the system clipboard, so that it can then (if this is desired) be opened in a traditional browser of the reader/player's choice (figure 16).
Figure 16: Opening a traditional browser with a link from News Reader.
If a story is displayed in a News Reader window, links appear within it (figure 17a). As with Regime Change, these links don't lead to other web pages, but rather generate texts out of a statistical text model (in this case, trigrams of the alteration corpus and their relative frequency). These generated texts appear in a new News Reader window (figure 17b). The alteration corpus is created from the texts of alternative news stories (found at Common Dreams) downloaded in the background when News Reader is launched. As with Regime Change, windows of generated text contain several paragraphs, each of which is a continuation of an n-gram that begins with the words in the clicked window just prior to that word clicked. And, again as with Regime Change, clicking a non-highlighted word will close a window of generated text, replacing the words clicked to open that window (figure 17c). The words used to perform this replacement will be those between the clicked word and the opening of the alternate text ("paragraph") it was within.
Figure 17a-c: A news story, a window of generated text, and a textual replacement in News Reader.
Playing Regime Change can produce a wide variety of texts, especially if one opens and collapses many layers of windows. The potential variety of texts created by News Reader dwarfs this. In fact, given that News Reader employs materials that change several times an hour, it may be more sensible to discuss the fact that no text created with it is likely to ever be repeated, rather than the fact that it can create a great variety of texts. But variety is still worth mentioning because, as Markku Eskelinen points out in "Six Problems in Search of a Solution," we struggle with a "deep rooted humanistic fear of variety"[46].
Eskelinen observes that this fear of variety, coupled with our need to grapple with digital objects that produce great variety, may be part of the attraction of computer game studies for some scholars. In computer games variation is pacified by rules and goals. Expanding this, Eskelinen writes that when we have lost "the safe and somehow manageable totality, be it coherent or not" we reach a point where "computer games are interesting, as they domesticate the excess looming large in both ordinary and avant-garde products and processes, and the fundamental potential for change and unreliability inherent in new media objects. "Perhaps this is also the attraction of performance, of instruments, as a way of talking about digital objects that produce great variety. Performance, and especially improvisational performance, is different each time — and yet we understand that it is structured. Perhaps this is also the attraction of discussing work in terms of playability — in terms of the potential of, and structures for, play.
In a different section of the essay Eskelinen puts his finger directly on the challenges facing those interested in creating playable texts:
"In any case an instrument is supposed to shape and frame the player's action and to produce interesting variation. This is a challenge that goes far beyond the overly hyped problems of non-linear presentation. As in any economy of means and ends, it is important to find suitable goals and patterns of change and variation in the functional and causal framework."
It is my hope that Regime Change and News Reader on some level accomplish this — providing suitable goals in the production of interesting textual experience, and simple patterns of change and variation based on n-gram logic and interactor selection. It is my hope that they can serve as early steps toward the development of a vibrant area of playable textual experiences, operating along logics more linguistic than graphical.