Agata Waszkiewicz is a Ph.D. candidate at Maria Curie-Skłodowska University in Poland. Their research focuses on experimental and metafictional videogames, as well as the representation of marginalized identities in video games. They published in several journals, including Game Studies and Journal of Games Criticism (in print). They enjoy experimental and narrative-driven games with original visual aesthetics and they especially appreciate games developed by Funcom, Red Thread Games (especially The Longest Journey 1999, and Dreamfall Chapters 2014), and Thatgamecompany (Journey, 2012).

In A Multimodal Approach to Video Games and the Player Experience, Weimin Toh describes the process that led him to develop a new, exhaustive model of the relationship between storyline and gameplay in video games. He synthesizes many of the theories of narrative game studies’ through the lenses of the multimodal approach, basing them heavily on the idea of ludonarrative dissonance as introduced by Clint Hocking in 2007. Grounding his research in the previous works and acknowledging the controversies regarding the concept, Weimin Toh does not concentrate solely on ludonarrative dissonance but rather regards it as one of a few types of relationships between two game modes, next to ludonarrative resonance and ludonarrative (ir)relevance. By further embedding his model in a “functional ludonarrativism” (Ryan 2006), Toh emphasizes that the meaning of the game is shaped not only by the text and the choices of the designers but by the player’s agency and experiences. Thus, the ludonarrative potential of the game cannot be realized without the interplay of all these elements. In the book, Weimin Toh reworks and expands upon the notion of ludonarrative dissonance, reframing it as a model rather than a single concept to show all the nuances and complexities of the issue. By using the multimodal approach he brings a new light into the discussion on the player-game relationship, in the end offering a model that can be an extremely useful tool for many game scholars.

Using multimodal approaches as his main methodological framework (Chapter 2) is Toh’s most innovative contribution to the field. Multimodality “focuses on how meaning is made through the use of multiple modes of communication as opposed to just language”, and “our meaning-making of the elements (and modes) within the text as a whole” (Toh 21). Due to its use of semiotic and linguistic frameworks, Toh’s book stands out from the significant amount of game studies derived from humanities research, especially film, literary studies, and philosophy. This background in applied linguistics and social semiotics informs the choice of the methodology and the attention to detail in the creation of the multi-faceted model. However, although the title of the book suggests a focus on multimodality, the second chapter is the only one that explicitly touches on the subject, only for the book to turn to narratological theories in the latter sections.

Chapter 4 introduces its first and most important concepts. Ludonarrative dissonance is a concept that was initially developed in order to point out the lack of consistency in how narrative, action, big-budget video games create and communicate their narrative through the story and gameplay. In its most popular examples, the scholars used it to criticize the BioShock series (Irrational Games/2K, 2007-2014) and Uncharted (Naughty Dog, 2007-2017) series in which the story and gameplay showcased different, and sometimes contradictory, behaviors of the protagonists. Although some (e.g. Swain 2010, Dunne 2014) voiced concerns about ludonarrative dissonance becoming a main tool of critiquing AAA games, Toh is far from demonizing it. On the contrary, he sees the potential in exploring the interactions between the story and the gameplay to help developers create meaningful and gripping narratives.

The book distinguishes three main ways in which ludonarrative dissonance can occur: narrative dissonance, information dissonance, and logical inconsistencies. In Toh’s typology, the narrative dissonance seems to be the closest to the original definition by Hocking, who critiqued BioShock (2K Games, 2007). Hocking argued that the game gives the player conflicting feedback, encouraging them through gameplay to act in self-interest, as opposed to the themes of selflessness shown in the story. Information dissonance is perhaps more subtle, as it refers to the conflict between the portrayed narrative—the player’s interpretation of the events—and the way in-game information is provided. In these situations, the player might be given contradictory information on what they are expected or able to do. This can occur, for example, when the game’s achievements encourage transgressive and disruptive behavior that is not condoned by the game’s story—think, for example, about NieR: Automata’s (PlatinumGames, 2017) “Wait! Don’t Kill Me!” achievement for killing machine lifeforms that have been established in-game as not only friendly, but pacifist. Thus, this type of dissonance can have interesting consequences to the player, who, unable to follow both the storyline and gameplay’s directives, must choose one over the other. Finally, logical inconsistencies relate to differences between the portrayal of character qualities in cutscenes and in gameplay, for example when certain characters are introduced as powerful foes in the cutscenes but prove to be easy to defeat. By noticing differences between different types of dissonance, Toh shows that part of the confusion around ludonarrative dissonance stems from the lack of precision when discussing it. Furthermore, by breaking the term down into separate sub-categories, he re-conceptualizes it as an umbrella category for a number of more specific problems that can be encountered by the player.

Overall, the book places more attention on the positive and neutral relationships between the narrative and the gameplay over the negative. In Chapter 5, once again using existing research to propose an elegant, if complex, structure, the author discusses the concept of a ludonarrative resonance (Watssman 2012, Brice 2011), also known as ludonarrative harmony (Pynenburg 2012). As opposed to the better-known concept of dissonance, this resonance refers to “the extreme fit between gameplay and narrative where gameplay is necessary to enhance the plot, and vice versa” (75). Resonance occurs when there is an integration of game objects, character feature(s) or actions and movements with the narrative setting of the game. The illusion of agency over one’s own actions in a game tends to correlate with the satisfaction derived from play. The chapter examines several mechanisms, including motivation, guidance, (semiotic) metaphor, balance, consequence/contingency, causality, succession, parallelism integration, prominence, player-character resonance and information solutions. All these categories have a double application. First, they provide a way to further conceptualize the types of players’ engagement and provide an explanation of the different reactions specific players might have towards the same title. Second, they can potentially aid game developers in creating an experience that facilitates immersion and satisfaction in players. While this analysis might seem tedious to some, it serves as a reminder that even the concepts that are considered well-known should not be exempt from careful scrutiny.

Chapter 6 dissects the last category of Toh’s model: ludonarrative (ir)relevance. Toh again borrows from Watssman’s theory (2012), to expand on it, and reject its original strict separation of the gameplay and the storyline. Instead, Toh argues that gameplay and narrative can and should be considered as situated on a spectrum, rather than in binary terms. Unlike the categories of dissonance and resonance, which are defined either through the coherency between modes or lack thereof, the (ir)relevance occurs when narrative and gameplay “have a weak relationship with each other” (99). Incomplete feedback and the lack of clear motivation for the characters can make it difficult for the player to immerse themselves in the game’s narrative. Some of the types of the (ir)relevance listed here include gameplay focus (where the player primarily concentrates on the gameplay, ignoring or losing interest in the narrative), information problems (when due to the incomplete information the players are unsure of the outcome of their choices) and metaphor (when the players cannot interpret the game’s feedback). Some of these, like guidance and prominence, parallel the ludonarrative resonance by showcasing situations where games fail to deliver clear feedback to the players. Interestingly, many types of (ir)relevance cause frustration in the players, which in turn causes them to turn away from either the storyline or the gameplay. What might be most helpful for developers is the realization that many of the discussed sub-types can either stem from oversight or be implemented purposefully, to deceive and manipulate the players—consider here the example of Spec Ops: the Line (Yager Development 2012), which forced the player to reflect on the brutality of war through metacommentary, fourth-wall breaking loading screens, and so on (Keogh 2012).

Having concluded the description of the model, the rest of the book concentrates on methodology and possible applications. Chapter 7 offers an explanation of Toh’s research, including the method of choosing the participants and the data collection process (such as recording gameplay and conducting open-ended interviews). Considering the importance of the individual experiences of the players who participated in the study, discussing methodology this late in the book is surprising. Throughout the book, Toh emphasizes the importance of the cooperative model of research in which the empirical data (often in the form of the recorder playthrough and interviews) is collected from several external players treated as “expert player co-researchers” (119). Since this method allows a researcher to collect a larger sample of data, it is possible to confront theoretical frameworks and hypotheses with diverse experiences, strategies and styles.

Chapters 8-11 take a step back to ground the model in the theoretical approaches with the focus on narrative and gameplay. While offering incredibly thorough research, these chapters give readers necessary theoretical context and would have made a better introduction to the model than a follow-up to it.

Chapters 8 and 9 discuss overall narration, with the focus on the players’ mental models (Chapter 8) and video game narrative analysis framework (Chapter 9). Both develop models in the broader context of games, discussing the possible reasons for the differences in gameplay experiences among the participants and showing how the model can be applied to other titles. The majority of Chapter 8 defines and expands on Marie-Laure Ryan’s concept of cognitive maps or mental models of narrative spaces (2003), which, by utilizing such elements as inventory, spatial relations, or the mapping style, enable the players to conceptualize and visualize the video game spaces. Weimin Toh recognizes that these models are additionally formed by the observation of not just the objects but the character behaviors, including their movements, actions, and the dialogue; this allows for the addition of the two elements: interactive character movements and interactive character movement with action. All these features, with the special emphasis on the two new categories, help players in understanding the character portrayal that informs the plot progression.

In Chapter 9, Toh moves to the discussion of various types of multi-linear narratives. He explores the various frames used for meaning-making, such as action, telling, experiencing, viewing, and reflecting (Fludernik 2003). Two innovations to the video game analysis model include the frames of manipulation and augmentation, which rely on recognition of player’s involvement with the game, unlike the original framework. While manipulation describes the control over the character’s movement, the augmentary frame includes the embedded narrative and “the layers of information, interpretation, backstory, and contextual frameworks concerning the game” (160).

The last major section of the book in Chapters 10 and 11 covers gameplay: first, from the perspective of the influence of player’s actions and motivations, and second, through the detailed analysis of its structure, including phases and feedback. The detailed description of various player actions in Chapter 10 is heavily embedded in four theoretical models: Montfort’s Interactional Framework in Interactive Fiction (2003), Heaton’s highly abstract Circular Model of Gameplay (2006), Manninen’s Interaction Forms and Communicative Actions model (2003), and Fabricatore’s Model of Gameplay Mechanics (2007). Chapter 11 continues to investigate the relationship between the player and the mechanics, especially in recognition of their ergodic actions and the feedback stimuli (using especially Oxland’s framework) as well as other macro components of the gameplay analysis model.

A Multimodal Approach to Video Games and the Player Experience is a highly specialized book that pays tremendous attention to its theoretical frameworks and methodology. Thus, it perhaps will be of more interest to game scholars rather than the casual non-academic readers. While I believe that the structure of the book could have benefited from the introduction of the methodology and the longer introduction of the overall theoretical background before moving to the model, its information density makes it an important ally for researching the player’s engagement, the relationships between the different modes of the narrative, or the application of the multimodal methodology. Furthermore, it shows that the ludonarrative dissonance is not an outdated concept, but rather an under-researched one.

Bibliography:

Dunne, Daniel. “Multimodality or Ludo-Narrative Dissonance: Duality of Presentation in Fringe Media.” Proceedings of the 2014 Conference on Interactive Entertainment, 2014, pp. 1–4.

Fabricatore, Carlo. Gameplay and Game Mechanics Design: A Key to Quality in Videogames. Proceedings of OECD-CERI Expert Meeting on Videogames and Education. Santiago e Chile, Chile, 2007.

Heaton, T. A Circular Model of Gameplay. Gamasutra. 2006.

Keogh, Brendan, et al. Killing Is Harmless: A Critical Reading of Spec Ops: The Line, Stolen Projects, 2013.

Manninen, Tony. “Interaction Forms and Communicative Actions in Multiplayer Games.” Game Studies: The International Journal of Computer Game Research, no. 3, 2003, p. 1.

Montfort, Nick. Twisty Little Passages: An Approach to Interactive Fiction. Cambridge, MA: The MIT Press, 2003.

Pynenburg, Travis. Games Worth a Thousand Words: Critical Approaches and Ludonarrative Harmony in Interactive Narratives. University of New Hampshire, 2012.

Ryan, Marie-Laure. “Cognitive Maps and the Construction of Narrative Space.” Narrative Theory and the Cognitive Sciences, edited by David Herman, CSLI Publications, 2003, pp. 214–42.

Swain, Eric. “In Defense of Ludonarrative Dissonance,” The Game Critique, August 5, 2010.

Watssman, Jeremy. “Essay: Ludonarrative Dissonance Explained and Expanded.” The Escapist, 2012.