Playing with Sound: A Theory of Interacting with Sound and Music in Video Games.

Playing with Sound: A Theory of Interacting with Sound and Music in Video Games. Karen Collins. MIT Press, 2013. 200pp. ISBN 9780262018678. Hardcover.

When Pac-Man dies, there's a cartoonish, descending tone-spiral that signifies his fate. It's also a sound of failure, made more bitter by the common role-playing conceit of games: you are Pac-Man, consuming white dots and fleeing ghosts in a maze, accompanied by auditory feedback and ambient noises. In a way, Pac-Man's sounds are your sounds, no matter how artificial they seem. It's probably a truism to say that sound in video games has come a long way from the "beeps and boops" of the 1970s and '80s, when arcade games like Pac-Man were popular. But even those old games featured something novel and special: anyone with a quarter could press a button to enter a world of images and sounds far different from those of daily life. Sound in video games thus entails more than listening; by playing with games, we also play with their sounds.

The technology and design of those sounds was the subject of Karen Collins' previous book, Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design (MIT Press, 2008), a valuable resource for both scholars and enthusiasts. The book helped ignite the field of game sound studies, which was then finding footholds in articles, chapters, and dissertations published across a range of fields. (1) Since then, research on video game music has gained traction in several new monographs and edited collections, and it is a consistent presence at major academic conferences. (2) It would not be an overstatement to say that Karen Collins' work has shown that video game music, whether produced by beeping synths or sweeping strings, is worthy of study.

Playing with Sound: A Theory of Interacting with Sound and Music in Video Games serves as both an update and a companion to Collins' earlier book, and in many ways raises the analytical stakes of the first volume: knowing that video games offer a uniquely interactive medium for creative sound design, what does it actually mean to interact with and among all of these sounds? Collins is interested in games as practices rather than texts, aiming to understand "the ways in which meaning is found, embodied, created, evoked, hacked, remixed, negotiated, and renegotiated in the space of interactive sound in games" (1).

From this ambitiously charted point of departure, Playing with Sound reaches far in its 150-or-so pages. Collins begins with theoretical accounts of sonic experiences in games, later emerging to discuss "real world" activities that draw on and extend those experiences. For Collins, "interaction" should be understood broadly, extending from physical engagement with a game to the interplay of elements on screen, the construction of interpersonal relationships among players, and the influence of larger sociocultural contexts.

Given the complexity of such a multifaceted notion of interaction, one of Collins' contributions is some helpful terminology. To characterize the action-sound interaction that occurs during gameplay, she adapts Michel Chion's concept of synchresis, which describes the fusion of sound and image in film, adding to it the term kinesonic. (3) The notion of kinesonic synchresis allows Collins to account for player-driven events that correlate with sound, which offer the player agency in sound-making--the boing of Mario jumping in Super Mario Bros. is only heard when a player presses the right button. Following Chion, Collins sees these moments of multimodal congruity (or incongruity) as having potential for new associations--"haptic recontextualizations"--that enhance the gameplay experience (39).

To conceptualize these experiences, Collins frames her study in terms of embodied cognition, drawing on work in psychology, cinema and media studies, philosophy, and human-computer interaction (among other fields). While the role of embodied experience might aid our understanding of a wide range of media, physical engagement is central to playing video games, and Collins argues that phenomena of "involvement, interpretation, and therefore attention" are implicated more directly (22). Following research on the mirror neuron system, Collins argues that sound's associations with image and action persist even if image and action are removed, and that these associations are embodied: hearing a sound associated with an action activates many of the same neurons that fire when the action is actually performed. Perhaps most intriguingly, because events in a video game are tied to player interaction, our embodiment might extend to rather complex actions our character performs, rather than ending with the simple button-press that triggered it.

Collins relates these embodied engagements to the various attentional and perceptual phenomena of "getting into" a video game, which can refer to feelings of immersion, incorporation, or character role-play. Game sound, Collins argues, facilitates these effects because it has the unique capacity to blend with sounds in the player's immediate environment, bridging the gap between player and game. She also suggests that it can help us better identify with in-game characters: if sound activates somatosensory neurons related to events, and those events are also tied to emotion and affect, then sound might also facilitate a deeper sense of empathy. As a consequence, we might adopt not only the actions of an avatar but also its goals, feelings, and personality. Collins mostly discusses voices and sound effects here, but we know that music can also be experienced bodily and engender emotions. How, then, might the musical sounds that are such an important part of many games also support different kinds of engagement and character identification?

While sound might facilitate a deep engagement with a game, its worlds, and its characters, sound can also allow opportunities for creativity and community building. These practices are the focus of the latter half of Playing with Sound. Music-based games like Rock Band or Just Dance turn players into performers by emulating spaces for physical and musical expression, like a rock concert or a dance club. Even non-performance games can offer technologies for customizing one's own in-game sonic experiences--for example, VoIP allows vocal communication over a multiplayer game network, which turns the soundscape into a social space for personal expression. Collins sees these social opportunities as redefining our notions of audience and spectatorship. Drawing on Kiri Miller's influential work in this area, Collins notes that in-game performance blurs our definition of liveness. (4) Because video games offer separate virtual worlds and identities, the performing body is distributed between the virtual and human spaces. To add to this multiplicity, game sound performance outside of games is increasingly popular. Collins describes a variety of media here, from parody videos, remixes, and large-scale concerts, to the development of new genres like machinima and chip music that utilize game technology to creative ends.

In the final chapter of her book, Collins turns to the various means by which players re-shape games, either by modifying pre-existing elements of a game or by creating new content. Changes such as these illustrate the greatest level of agency that a player can have over her or his play experience, sonic or otherwise. Collins also suggests that modifying games can be a way to extend the gaming experience rather than a form of creative resistance. Although the various kinds of interactions described here don't necessarily involve sound, they often do; playing with sound is fun. And part of that fun is the game players' practices of co-creation (of experience, meaning, and even content) with game designers and fellow players.

In the few years since the publication of Playing with Sound, forms of interactivity in video games have multiplied, particularly with recent developments in virtual and augmented reality devices. Nonetheless, Playing with Sound should have staying power as a central resource for scholars interested in games and sound studies. Collins has written a well-researched book, brimming with multidisciplinary sources at every turn. Its brisk pace keeps many topics afloat, though the reader might be left wanting some breathing room for one topic to develop before moving along to the next. But she does acknowledge that the book asks more questions than it answers, and Playing with Sound situates itself as a launching point for more discursive treatments of the many issues it raises. From this perspective, Karen Collins achieves breadth, depth, and clarity--a special kind of interactivity, indeed.

Julianne Grasso, University of Chicago

Notes

(1) Game Sound was likely the first book not aimed at aspiring game sound designers. One of the first and most cited contributions to the scholarly realm of game sound was Zach Whalen's article "Play Along: An Approach to Videogame Music," Game Studies: The International Journal of Computer Game Research 4/1 (November 2004), http://www.gamestudies.org/0401/whalen/. The website for the Ludomusicology research group contains a helpful and frequently updated bibliography.

(2) Some recent book publications on the topic of sound in video games include William Cheng, Sound Play: Video Games and the Musical Imagination (New York: Oxford University Press, 2014); K.J. Donnelly, William Gibbons, and Neil Lerner, eds., Music in Video Games: Studying Play (New York: Routledge, 2014); Michael Austin, ed., Music Video Games: Performance, Politics, and Play (New York: Bloomsbury Publishing, Inc., 2016); Michiel Kamp, Tim Summers, and Mark Sweeney, eds., Ludomusicology: Approaches to Video Game Music (Sheffield, UK: Equinox Publishing, 2016). Additionally, 2012 marked the first Ludomusicology conference in the UK, and 2014 welcomed the first annual North American Conference on Video Game Music, at which Karen Collins gave a keynote talk.

(3) Michel Chion, Audio-Vision: Sound on Screen (New York: Columbia University Press, 1994), 63.

(4) See Kiri Miller, Playing Along: Digital Games, YouTube, and Virtual Performance (New York: Oxford University Press, 2012).

Article Details

Printer friendly Cite/link Email Feedback
Author:	Grasso, Julianne
Publication:	Society for American Music Bulletin
Article Type:	Book review
Date:	Jan 1, 2017
Words:	1595
Previous Article:	Student forum in Montreal.
Next Article:	Remixology: Tracing the Dub Diaspora.
Topics:	Books Book reviews