Becoming Sensate

New Approaches to Observing Play

Krystin Gollihue is a PhD student in the Communication, Rhetoric and Digital Media program at NC State University. Her research explores feminist approaches to critical making that account for the deeply historical and multimodal ways women have created communities of work, literacy, and communication. bio-twitter

Introduction: The Apparatus of Play

Games move us (Apperley & Jayemayne, 2012; Giddings, 2009). They teach us how to play them, how to move through their worlds, how to learn their protocols, and how to negotiate their persistent blending of virtual and physical worlds. The moving parts of a situation of gameplay – the platform, the narrative, the player and her environment – act as an assemblage, a constantly changing interaction of humans and nonhumans influencing one another. The notion of play is not a fixed reality, but a result of these elements in constant contact and becoming what we recognize as play (Massumi, 2002).

As I’ve recently become both a gamer and a researcher of games, I have come to understand the gaming assemblage as more than just the screen or console in my hands. I use field notes and digital diaries to reflect on my gaming experiences. I use my eyes and my body, or sometimes a recording device, to make sense of what I’ve just accomplished or learned. Each of these instruments of measurement affects what I come to understand as play; each is a “dynamic (re)configuring of the world” (Barad, 2003, p. 816).

At the end of 2016, I took on a project called “Becoming Sensate” that aimed to observe observation, to uncover what my play became when I paid attention to how I made sense of experiences. What happens when we watch our play, when we capture it, when we measure it? During “Becoming Sensate,” I measured my heart rate during play sessions, affixing an Arduino-powered pulse sensor to my body and identifying any patterns or changes in the heart rate data as I played two games with two very different emotional tones, Fallout 4 and Minecraft. Taking discursive notes during sessions, I looked for how the pulse sensor configured me into certain postures or how the data changed as I encountered different phenomena in and out of the game.

A pulse sensor graph

When the sensor comes on, I become both player and measurer (Lachesis the fate of weaving), but I’m not quite certain yet what I’m measuring. Am I measuring my affective response to, like, hacking away at what sounds like puffy cubes fashioned like cobblestone? Or am I measuring the game itself? It isn’t representing my heartbeat, or by extension my psychic state. But could it actually be configuring my physicality? By playing this game with this monitoring device, am I forging a specific way of being in the game as a neurodivergent person?

This study was not simply a gamer researching herself playing games with a sensor attached to her ear. It was a communing: an intimate exchange between human and nonhuman actors and ambiences. My sensor left traces of itself in the ways I moved around the house after play, slightly tense and feeling the ghost of a microcontroller against my ear. Measurement configured me in specific ways, allowing me to experience specific realities, specific situations of play. “Becoming Sensate” was less a measurement of experience than it was an experience of measurement.

Apparatuses of Measurement

The term data – what we collect when we do research – refers to “something that one gathers, hence is a priori and collectable” (Markham, 2013). Co-opted by early social science research, positivist methods of observation “focus on predicting and controlling ‘behavior’, and the use of quantitative methods in artificial settings” (Lindlof & Taylor, 2011, p. 6). Through positivist approaches to knowledge, we come to know a fixed and already-existing reality via measurement. In contrast, qualitative methods such as life histories and verbal narrative describe reality as plural and emerging – as an individual instance rather than a representation of the whole. Measurement is aimed at capturing and collecting the data of a situated moment – an interview or an interaction.

Barad (2003) however, argues that measurement itself creates the conditions for both positivist and emergent realities. Our instruments read back our data to us (Markham, 2013) and have particular ways of framing our world. Positivist instruments used in artificial settings not only predict and collect already-measured realities; they create (or recreate) already-measured realities. A participant or phenomenon has no choice but to be predictable when the apparatus of measurement can only act in those specific ways. For example, in the case of the pulse sensor, a heart rate can only be described as high (unhealthy), low (unhealthy), or normal (healthy). These conditions for reality, as Barad argues, mean that a person whose heart is being monitored can only exist within this binary of sickness and health.

Scientific studies that use the pulse sensor as an observation tool describe it as observing a more perfect reality (González et. al, 2014; Schubert, D’Ausilio and Canto, 2013). A more perfect reality however, has far-reaching implications for research in game studies. A more perfect reality is an outcome of intra-action (Barad, 2003), or the components of a phenomenon interacting within itself. If we are to understand games and gaming assemblages as vast intra-actions of phenomena, then we must understand our scientific instruments of measurement as an integral part of that meaning-making.

We must consider the realities that are being made, and made perfect, by a tool like the pulse sensor. We must also consider what is not being revealed, what is being erased even, by quantitative methods in artificial settings. Is there something else we can uncover when we consider quantitative methods in situ, in natural settings? What happens when see our tools of analysis as agents capable of being contextualized? In the following section, I aim to answer this question by describing the complex intra-action of play and measurement during “Becoming Sensate.”

Reconfiguration: The Creation of a Playing Body

During “Becoming Sensate” I played two games for my observations: Fallout 4 and Mojang’s Minecraft. Released by Bethesda in 2015, Fallout is an emotionally costly game: the very first play sequence imagines a nuclear holocaust, the loss of a child, and survival from mutant hostiles. A single 4-hour session going between the slow, silent crawl of movement and the frenzied violence of action felt more like work – exhausting, emotional work – than play. To contrast the first-person shooter experience with something more light-hearted, I also played Minecraft, a procedurally generated sandbox game where the goal is to explore unique worlds, craft items, build structures, and (optionally) fight off foes.

A pulse sensor graph

I power on R’s Xbox One, watch the Fallout 4 Vault-Tec load screen circle on the screen as I attach a flimsy heart rate sensor to my right earlobe. I’ve been told the ear is an appropriate place to find the human pulse, so I clip on the light-dependent phtocell sensor to my ear like costume jewelry. I’ve been watching the #NoDAPL news all day. The Dakota have drones now. I’m exhausted. A suit of power armor appears in a blown-out workstation somewhere near Concord, Inon Zur’s “The Last Mariner” playing. R’s wife, J, is preparing dinner in the next room, chicken smell somewhere in the house. R laughs, “You really are turning into a cyborg, aren’t you?” […] With Dogmeat by my side, I turn on Diamond City Radio. It’s Billie’s “Crazy He Calls Me.” I lock in a fusion cell and listen for the Deathclaw outside.

I found throughout the study that the line between action games like Fallout and more casual games like Minecraft was not a clear boundary. Casual games are defined as “structures of feeling” that “work on us and work us over…impinging on our feelings, our identities, and our everyday lives” (Anable, 2013, p. 3). I perceived Minecraft as fitting this definition, given its lack of prescriptive narrative and my ability to choose not to kill things, to walk through the game world and pick flowers or build houses. However, Fallout also impinged on my feelings, albeit in different ways: it asked me to pay attention to the narrative elements of a post-apocalyptic world and weigh my choices in survival and death, but I also could spend days modifying my arsenal and gathering weeds from the land for food. While most of my experience with Fallout was marked by negative feelings of fear and anxiety, Minecraft asked me to configure myself as a destroyer of the environment, something that also made me feel concern. There was something relaxing about digging up dirt cubes, until I opened up a cavern and plummeted to my death. In both of these games, there were configurations of different elements that blurred the line between casual and active, giving me an experience of gameplay as more generally a moment of intensity. That “structure of feeling” could be any feeling, not necessarily positive or negative, active or relaxed.

These structures of feelings – the emotional labor – of both games were in many ways configured by the pulse sensor itself. In these sessions, configured like a cyborg with an extrasensory attachment, my whole body was primed for a danger/fight-or-flight response. The sensor is a small component with an LED fashioned to be a clip on the end of a finger or the tip of an earlobe. I clipped it tightly (and painfully) to my ear so as to keep it attached during especially active moments of play. The sensor was then attached to an Arduino board, which was strapped to my shoulder using medical tape wrapped around my upper body. The wrapping, which acted as a binder, often cut off circulation to my extremities. In order for the whole apparatus to stay in place, I had to play with a hunch to my back and with my right shoulder close to my ear. The sensor constrained me, numbed me. It configured me as uncomfortable, fearful, tense, even pre-mammalian in my need to respond to threats.

A pulse sensor graph

I watched my spouse get killed and my baby stolen away. Then I woke up from the cryopod, and crept around the devastation of the vault. Between now and the time I started, I heard the buzzing of mutant cockroaches, saw the blinding light and absolute emptiness of my hometown post-fallout. There have been moments of silenc eand moments of chaos. There is the moment when I rise from the dark vault to the ashy surface of the earth and I see the devastation for the first time. Somber cello music couples with a single hawk flying over the blasted out remnants of the vault site. Not long after, giant mutant mole rats erupt out of the gorund near an abandoned gas station. It’s either loud or empty, fast or slow.

I see these experiences as a kind of proprioception made possible by the sensor, a new kind of organ that “makes feelings feel” (Shouse, 2005, p. 2). The entire contraption strapped to my body brought about a heightened sensitivity to emotion, to pain, and to feeling. I was permeable with my sensor on, always in response to the measuring apparatus and to some aspect of the game. The Enderman’s long, slow moan became louder because there was a pinching at my ear. The sad silence of my avatar’s hometown in Fallout was that much more deadening because of the numbness in my arm. Particularly during moments of intense feeling during play, I was captivated and captured by my sensor; I paid attention to the game in ways that were informed by my body’s interaction with my tool for observation.

The Agency of Data

While the sensor-as-object had a tremendous effect on my playing body, the numbers the sensor registered also played a significant role in the gaming assemblage. In one sense, higher numerical data represented moments where I was surprised or frightened, and lower data indicated moments of inaction or relaxing play. Numbers stood in for play, for what happened to my body in the moment of play.

A pulse sensor graph

I killed a pig today. Its unreal, square little bug-eyed face, uncanny valley for sure. I took its little round pink meat afterwards and placed it in the furnace slot above a piece of coal. Plug and play slaughter. I’m pretty sure my sensor fell off when I started getting slammed by what I now know is an Enderman. Wishing I could have caught the data from what I was feeling when that glitchy voice started in. I don’t think I’ve ever been more afraid of something in my life.

However, these assumptions about what my data represented are more in line with a positivist approach to knowledge, where an instrument of measurement captures an observable and already-existing reality. Some things would have been erased from the record if I had taken the reality that the data configured at face value. For example, sometimes nothing was being captured by the sensor: the Arduino still registered data when it was not attached to my ear. Many of the drastic spikes in numerical data were not a result of my heart responding to a moment of surprise or fear; they described moments when the sensor was pulled off during especially boisterous or physical play sessions. Even when the sensor is not there, there are still realities being configured that never actually happened. Other ways of watching – taking notes, for example – created other realities, ones of disruption and frustration at the sensor falling off, ones in which the reality being configured was that of a relationship between me and my sensor, not between me and my gameplay. Numerical data could only describe a certain reality, and not necessarily one that was more perfect as a result of its technology.

Barad (2003) argues that measurements are not representations of things; they are things, phenomena that result in certain realities played out in certain terms. My data was therefore a thing in and of itself, a result of the particular interactions between body, game, environment, and most importantly measuring apparatus. Instead of approaching my data as abstract and separate from me and my sensor, I see it as an integral player in how my experience was configured as dangerous or safe, exciting or boring. My sensor had agency; it became apart of me, acting on me, influencing me to see and feel the game in certain ways and with certain words. It could not describe my experience of play; it only described my experience of being measured, of coming to terms with being watched.

Conclusion: Experiencing Measurement, Making Meaning

After several months of watching my sensor watch me play games, I have come to understand play as deeply implicated by the tools I use to measure, observe, and reflect. My sensor, my body, my field notes – all the instruments that I use to make meaning – have particular effects and affects on me and what ends up being my experience of play. The data are not always representations of what play actually is; sometimes it is simply a tool misfiring, a different kind of moment being drawn into truth.

Our experiences of play are also experiences of measurement, of our tools watching us and paying attention to us through our interactions with them. These are the undercover agents that form what we know as play, what brings us to the screen to enter into new worlds, new feelings, and new intensities. Our goal as researchers, developers, and gamers is to thicken our data (Haraway & Goodeve, 2000), to account for what is there that moves us, for what shapes our experiences. As we seek to reveal the hidden actors and factors in how our experiences of play are formed, we should look to the agencies of our measuring tools as yet another avenue for making meaning.

References

Anable, A. (2013). Casual games, time management, and the work of affect. Ada: A Journal of Gender, New Media, and Technology, 2. Retrieved 28 September 2016.

Apperley, T.H. & Jayemane, D. (Oct. 2012). Game Studies’ Material Turn. Westminster Papers in Communication and Culture, 9(1), 5-25.

Barad, K. (2003). Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter. Journal of Women in Culture and Society, 28, 801-831.

Bucher, T. (2012). A Technicity of Attention: How Software “Makes Sense”. Culture Machine, 13, 1-23.

Giddings, S. (2009).Events and Collusions: A Glossary for the Microethnography of Video Game Play. Games and Culture, 4(2), 144-157.

González, F.C.J., Villegas, O.O.V., Dulce Esperanza Torres Ramírez, D.E.T., Sánchez, V.G.C., & Domínguez, H.O. (2014). Smart Multi-Level Tool for Remote Patient Monitoring Based on a Wireless Sensor Network and Mobile Augmented Reality. Sensors, 14(9), 17212-17234. doi: 10.3390/s140917212.

Haraway, D. & Goodeve, T. (2000). How Like a Leaf: An Interview with Thyrza Goodeve. London: Routledge.

Lindlof, T. & Taylor, B. (2011). Qualitative communication research methods (3rd ed.). Thousand Oaks, CA: Sage.

Markham, A. (2013). Undermining ‘data’: A critical examination of a core term in scientific inquiry. First Monday, 18(10). Retrieved 16 January 2017.

Massumi, B. (2002). Parables for the Virtual. Durham: Duke University Press.

Schubert, T.W., D’Ausilio, A., & Canto, R. (2013). Using Arduino microcontroller boards to measure response latencies. Behavior Research Methods, 45(4), 1332-1346. doi:10.3758/s13428-013-0336-z.

Shouse, E. (Dec. 2005). Feeling, Emotion, Affect. M/C Journal, 8(6). Retrieved 28 September 2016.