Jason Hawreliak received his Ph.D. in English from the University of Waterloo. His research examines rhetorics of heroism and immortality in videogames. Other research interests include multimodal rhetoric and the psychological function of digital media. He is essays editor for First Person Scholar

Before our August hiatus, I wrote that game critics and theorists need to be timely and accessible if we’re going to influence the games industry and gaming culture in general. I believe in that approach very much, but our persuasive goals needn’t always be so pragmatic. It’s nice to exercise different intellectual muscles from time to time, and that’s what I’m doing here. In this essay, I’m interested in a very basic but difficult question: How do we extract meaning from a videogame? How does it signify to us, the players?

There are many ways to answer this, but a good place to start is by acknowledging that videogames are many different things at once. This idea is relatively old, but the best and most focused work on the medium’s multiplicity is Ian Bogost’s (2006) Unit Operations: An Approach to Videogame Criticism. In this work, Bogost reiterates the point that at any one time a videogame can be thought of as a piece of hardware, software, lines of code, equations, images, sounds, rules, a commodity, a work of art, a story, and so on. Bogost describes these component parts as “discrete units,” which do not exist in isolation, but rather, in a radically dynamic relationship with one another. It is important to recognize the distinctness of each unit, since in Bogost’s words, “Each medium carries particular expressive potential” (p. 15), but it is also important to recognize that these units cannot help but interact with one another.

To examine the interactions between discrete units, Bogost suggests a methodological approach he calls “unit operations,” which “strive to articulate both the members of a particular situation and the specific functional relationship between them” (p. 14). In emphasizing the relationship between discrete units, further avenues of meaning are opened up to the critic. For instance, instead of asking, “what are the narratological themes in a game?” we might ask, “how do the narratological themes interact with gameplay mechanics?” In doing so, this approach emphasizes (re)configurations, and therefore “meander[s], leaving [hermeneutic] opportunities open rather than closing them down” (p. 7). This form of analysis thus lends itself to a richer and fuller understanding of a videogame than a monolithic approach might.

Mode, Metaphor and Metonymy

One way we can organize these “discrete units” is in terms of modality. A videogame can be thought of as a mode in itself, but it is also comprised of several other communicative modes as well. What does and doesn’t constitute a mode is always up for debate and mode can mean many different things depending on the discipline, but in very general terms it can just be thought of as the way something is expressed. In cognitive linguistics, modes are often described in terms of sensory data, corresponding with the five senses. Thus, there is a visual mode (painting, photograph, cartoon), aural mode (music, sound effects), haptic mode (touch), olfactory mode (smell), and gustatory mode (taste). However, as Charles Forceville (2006) notes, these can then be broken down into at least nine communicative modes:

“(1) pictorial signs; (2) written signs; (3) spoken signs; (4) gestures; (5) sounds; (6) music (7) smells; (8) tastes; (9) touch” (p. 382).

Typically, a videogame utilizes (1) pictorial signs; (2) written signs; (3) spoken signs; (4) gestures; (5) sounds; (6) music; and (9) touch. Smells and tastes are not utilized right now in the commercial sector, but the US Army uses a vehicle simulator which also includes smell, used to treat PTSD.

For videogames, I’d add a few more modes of expression, though I’m mixing my disciplines here: (10) Narrative (plot, setting, character); (11) Procedurality (rules, limits, processes);  and (12) Ludicity (game type, reward structures). The question then becomes, how might videogames utilize this multimodality for signification?

One way I’d like to explore here is through metonymy. Metonymy is another complicated term and it has deep implications for how we perceive the world around us, but here I’ll treat metonymy as a subspecies of metaphor, wherein I view metaphor in the broad, Nietzschean sense, i.e. the substitution of one thing for another across two domains, the material signifier (vehicle or source) and the intended meaning (tenor or target). Metonymy is also a substitution of sorts, but the relationship between the vehicle (source) and tenor (target) is an associative one. For example consider the following metonymy:

“The WHITE HOUSE sent out a press release condemning the Assad regime.”

The actual building of course didn’t send out a press release, but we automatically associate WHITE HOUSE (source/vehicle) with the PRESIDENT, or U.S. GOVERNMENT (target/tenor). A different, but perhaps more pertinent example of metonymy is Pavlovian conditioning. In one experiment, Pavlov famously rang a bell whenever he would feed a dog, and eventually got the dog to associate BELL with FOOD so that when Pavlov rang the bell, the dog’s stomach would growl. For the dog, BELL (source) became inextricably linked with FOOD TIME (target).

For my purposes, all that’s important is that metonymy is an associative link, and most importantly, that this associative link depends upon the prior experience and knowledge of the audience. Presumably, if you said “The White House sent out a press release…” to an “uncontacted” community in the Amazonian rainforest, they would not understand the metonymy. As Hugh Bredin (1984) puts it, “metaphor creates the relation between its objects, while metonymy presupposes that relation” (p. 57).

According to Charles Forceville, the leading figure in work on multimodal figuration, a multimodal metaphor is “a metaphor whose target and source are not, or not exclusively, rendered in the same mode” (2007, p. 19). An example of a multimodal metaphor in a cartoon would be squiggly lines (visual) to signify that something stinks (smell), or maybe that it is hot (touch).

In Downing and Mujic’s (2011) words, “This definition can be applied to the analysis of multimodal metonymy, which similarly involves the presence of source and target domains in metonymies in at least two different modes” (p. 157).  To give some examples of multimodal metonymy, I’ll apply these concepts to constructing otherness in Western shooters.

Multimodal Metonymy and Otherness

Again, metonymy only works by drawing on the prior (“presupposed”) knowledge of the audience. With this in mind, I’ll now examine how most contemporary Western shooters signify enemies by drawing upon established metonymies. First, in the abstract sense enemies are often signified by using the colour RED in some manner, such as a red dot on a radar, the enemy’s name written in red (e.g. above their heads), and so on. Similarly, allies are often signified using green in the same ways. The game does not need to explicitly tell you who is friend and who is foe; we just know automatically that RED means ENEMY and GREEN means ALLY. This is because we often associate RED with DANGER in our experiences, (e.g. traffic lights/signs, gauges, blood, etc.), and GREEN with GOOD. This is a multimodal metonymy since the visual mode (red dot) informs the ludic mode (opponent), and so we know to associate RED with ludic “danger” without being told so explicitly.

This is a fairly benign example, but what if the visual signifier for the (ludic) ENEMY becomes something more sinister, such as ethnicity? Given the generic conventions, encountering an armed Middle Eastern man in a Western Shooter, for instance, is likely to be associated with ENEMY. It would be perfectly reasonable to expect the man to begin shooting at you, though a game like Spec Ops: The Line (Yager/2K, 2012) plays on this precise expectation. If it was an armed white man instead, this might not be the case, (though it depends on clothing, stance, and other signifiers of course). There would at least be some more ambivalence and ambiguity when compared to the Middle Eastern man. Thanks to generic conventions which themselves are informed by broader cultural discourses, enemies in shooters tend to be from only a few regions in the world.

The ludic and procedural structures of a shooter are fairly consistent. It is almost always an agonistic game (competitive) where the player must “shoot” other players or computer controlled enemies. It’s kill or be killed, and whoever kills the most or captures an objective wins. So with very few exceptions, the genre is inherently competitive, where killing means winning. As Bogost (2007) remarks in Persuasive Games, there is no diplomatic option and no surrender. So because we’re used to this, right away we know that there will be enemies who want to shoot us and whom we’ll have to shoot. How these opponents are represented in the visual and aural modes means they will be metonymically linked with the antagonism and hostility of the shooter enemy generally. This is potentially harmful because it just plays into the general conception of Middle Eastern peoples, whom we often see depicted as angry, violent, irrational and fanatical.

To really drive the point home, let’s flip this on its head. Say you’re playing a shooter but when you encounter your first enemy it isn’t a Middle Eastern “terrorist,” but Clementine from Telltale’s The Walking Dead (2012) game.

Imagine all of the enemies in CoD looked and sounded like Clem. In every other way they would act the same, i.e. aggressive, reckless, inhuman. Depending on the context, we would probably find this either weird, disturbing, or humorous. And this is because the modal metonymies don’t agree with one another. Clementine’s youth and appearance conceptually map to “innocent,” for example; we associate youth and innocence with PROTECT, and so it would be quite jarring and go against our expectations if she were to aggressively charge you firing an automatic weapon.

To complicate it even further, also imagine that she wasn’t yelling (aural mode) as she charged, but saying “You’re dead, pal!” in a cute or sweet voice. It would be more than a little creepy, and this is at least partially because the metonymies are all mixed up and conflicting with one another here. We associate the enemies in a shooter (ludic mode) with aggression and unwavering hostility, so they can’t look or sound passive, vulnerable or innocent. So instead, these games have to rely on stock images and stereotypes which we automatically associate with violence. Unfortunately this too often takes the form of particular ethnicities, nationalities and even genders.

Conclusion: A Metonymic Feedback Loop

Games use metonymies by drawing upon the existing knowledge of their audiences; examining a game’s metonymies (multimodal or monomodal) can thus grant us greater insight into the culture from which they emerge. However, this also works in the reverse as well: Games rely on previously established representation, but as cultural artefacts they also contribute to strengthening the often harmful associations our culture has with particular ethnicities, genders, religions, and so on. Like other cultural artefacts games depend upon presupposed knowledge, but they also contribute to what is presupposed in the first place. Jarring examples of modal disagreement—such as Clementine with an AK-47—highlight this by exposing our already-held expectations and associations.

As I see it the danger is that repeated exposure to media that link ISLAM with TERRORISM, for example, or WOMAN with SEX OBJECT causes people to further internalize these metonymic relationships. Currently, games are part of the problem, but they don’t have to be. The recent rise in game diversity, mostly coming out of the independent scene, is a step in countering these harmful stereotypes and associations. Since videogames are a multimodal medium, examining modal relationships is important for understanding how they generate meaning. In my view, this means more work on the relationship between videogames, cognitive linguistics and conceptual metaphor theory in general. Broadening our disciplinary base can only strengthen our roles as effective critics and theorists.

 Works Cited

Bogost, Ian. (2006). Unit Operations: An Approach to Videogame Criticism. MIT Press.

Bredin, Hugh. (1984.) Metonymy. Poetics Today 5.1, 45-58.

Forceville, Charles (2006). ”Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research.” In: Gitte Kristiansen, Michel Achard, René Dirven and Francisco Ruiz de Mendoza Ibàñez (eds.), Cognitive Linguistics: Current Applications and Future Perspectives. Berlin/New York: Mouton de Gruyter, 379-402.

Hidalgo, Laura, D. and Mujic, Blanca K. (2011). Multimodal metonymy and metaphor as complex discourse resources for creativity in ICT advertising discourse. Review of Cognitive Linguistics 9:1, 153–178.