SHATTERING THE SCREEN: EMBODIED NARRATIVE IN DIGITAL MEDIA/SUDAUZYTI EKRANA: IKUNYTASIS NARATYVAS SKAITMENINESE MEDIJOSE.
Our society is awash in media storytelling. We would like to know how our experience of narrative in digital media is different from everyday life. How is our apprehension of stories affected by the medium of communication? How does the screen promote or inhibit storytelling in media such as video, film, computer games, or virtual reality web sites? Answers to these questions would help us to understand digital media better as well as ourselves. Granting that digital media offer sensations that are unique and unlike the other storytelling modes, then what follows is a modest attempt to delineate in precise terms the fundamental, distinguishing characteristics of the perceptual experience of narrative in digital media. Given the well-documented imbalance of the human senses in favor of vision (Arnheim 1974: 42-46; Jay 1993: 21-148; Wilson Nightingale 2004: 1-39; Ong 1958: 108-114) and dominance of vision in the digital media arts (Heilig 2001), this discussion is limited to the visual field, and experiencers are referred to as "viewers" (1). The visual field sets the stage, so to speak, for storytelling in all screen experiences.
This paper's method of inquiry is phenomenology, a science of human experience promulgated by European philosophers (Husserl 1962; Merleau-Ponty 2002) and applied to problems of human-technology interface by North American scholars (Ihde 1979; Pilotta, Mickunas 1990; Weber 1996). The phenomenological method is appropriate for this investigation for several reasons. First, phenomenology embraces phenomena as they present themselves to us, without presuppositions or privileged perspectives inherent in other analytical frameworks; the only framework for phenomenology is the structure of human experience itself. Second, phenomenology sets aside or "brackets" other methodologies, such as empiricism and psychologism, without prejudice or antagonism, in order to make its own unique epistemological contribution (Husserl 1962: sec. 31, 96-99; Pilotta, Mickunas 1990: 12). Third, phenomenological descriptions are rigorous and detailed. Fourth, phenomenology can decipher quandaries of human-machine interface. And fifth, Don Ihde has shown that phenomenology can be presented in plain language accessible to non-philosophers (Ihde 1983: 47-64).
Embodiment in everyday life
Embodiment means having a perspective or point-of-view toward the world, a simple idea with profound consequences. We are not conscious of being in our bodies or seeing from our bodies; we simply are ... here (wherever that is), with our awareness directed out there ... toward the world. Though eyeballs give us the visual impressions we have, we do not see in our eyes; rather, we see in the world (Stroker 1987: 144). For example, a mirror image of one's self is out there in the world, not in here where I am. Even persons with partial or total visual impairment have visions, which though different from sighted persons, are equally outward-directed toward the lifeworld and correlated to their everyday activities (Scholz 2008).
Our sensory experience of the world is represented by the expression (observer [right arrow] world). Self-awareness of having a point of view tells an observer that there is an embodied relation to the world; this reflection is represented by the expression, (observer [left arrow] world). In Maurice Merleau-Ponty's words, the lifeworld makes our visual experience possible and fulfills it (2002: 230). Thus, vision is a reciprocal, embodied relationship with the lifeworld: (observer [??] lifeworld). This basic correlation of observer and observed is true of the entire sphere of human experience, as represented by the all-encompassing ellipse in the diagram in Figure 1.
Two everyday topological examples can illustrate the idea of visual embodiment. The first example is the simple task of drawing the floor plan of a room (see Fig. 2).
Most of us would be able to draw a reasonably accurate overall shape of a room we are in (Merleau-Ponty 2002: 235). We would be able to sketch the room's features in correct spatial relation to one another. Such a performance is possible only because of our innate facility to embody an imaginary perspective above the room looking down on it, without being aware that we are doing so. We must embody the elevated point of view to draw the room floor plan. The other example of embodiment in everyday life is reading a city street map to compare one's heading in the real world to the marks on the map (see Fig. 3).
First, we compare street signs observed at an intersection to map labels to correlate our perspective of the street to the map. Next, we look for some additional landmarks to determine our looking direction on the map. Just as in the room example, the task of map reading requires embodying projected points of view.
The two preceding examples of visual embodiment entail comparing lifeworld perspectives to static pictures (floor plan, street map). The pictures themselves do not bring about embodiment of perspectives, which is a temporal process. Embodiment is a sort of imagined storytelling, having particular temporal order and duration. Screen media present changing spectacles that provide the requisite spatiotemporal transformations for embodiment of perspectives to occur. In screen media, embodiment can be complex and powerful for infinitely variable storytelling.
To illustrate the role that embodiment plays in screen media narratives, let us consider bicycling, an activity encountered around the world both on the street and in screen media. We may ask, how can we share the vicarious experience of riding a bike from watching a "virtual reality" cycling video? This question pertains to training videos used by cycling enthusiasts, such as the "45-Minute Indoor Trainer Workout" by CTXC Training Videos (2013). By watching this video, cyclists can enjoy a 40+ kilometer per hour ride along the Beach Road, Melbourne, Australia, while peddling their stationary bikes. The basic mechanism for the compelling grip of such "virtual reality" screen media on our imaginations is embodiment of shared perspectives.
We can share embodiment with each other through eye gaze. When we see other humans in the act of looking, we naturally imagine their visual perspectives, which are different from our own. Merleau-Ponty calls this facility apperception or seeing with another person's eyes: "We are collaborators for each other in consummate reciprocity. Our perspectives merge into each other, and we co-exist through a common world" (2002: 413). Herbert Zettl identifies eye gaze as the most powerful graphic force leading a film or video spectator's focus of attention (2010: 360). When the eye gaze of an observed person or other sighted figure aligns with the Euclidean Z-axis of the observer's visual field, it can help to articulate the observer's perception of depth in the scene (see Fig. 4).
Screen media offer a special case of eye gaze in visual experience, because the experiencer's own eye gaze aligns with and defines the Z-axis of the visual field, but also is locked into it. This situation can be thought of as a "borrowed" eye gaze in which the viewer shares in the spectacle of the medium (see Fig. 5). The mediated eye gaze of the screen introduces special transformations in embodied experience that are outlined in the next section.
Technological embodiment of media
Technologies can intervene in basic human-lifeworld relationships to extend a viewer's vision beyond normal limits. Still photography is a good example of visual technologies that provide what Laszlo Moholy-Nagy calls telehor or far-seeing (1967: 30). A photograph can make something from far away appear close and something of another time appear now. Technologies such as photography take a mediating position (hence the term "medium") between a viewer and the lifeworld, as in (viewer [right arrow] medium [right arrow] lifeworld). A visual medium such as photography embodies the world for a viewer by making the world present. Thus, technological mediation is a special subset of all embodied perspectives on the lifeworld and is symbolized by the first nested ellipse in Figure 1.
Ihde suggests that we desire our communications technologies to be transparent, or to recede from our awareness, leaving a simulacrum of a direct, unmediated encounter with the lifeworld (1983: 50). For example, a photograph that provides a clear, lifelike image of a subject allows a viewer to see beyond mediation to the photograph's subject as situated in the viewer's own lifeworld. A totally transparent medium recedes into the experiencing self, as in (viewer [right arrow] medium [right arrow] lifeworld). An example of a non-transparent medium is an out-of-focus, grainy photograph that partially obscures the subject. Lack of transparency prevents the viewer from seeing through the photograph at its subject and reminds the view of the technology's mediating position in the visual experience. Lack of transparency in mediated experience can be represented as a viewer's reflexive awareness of the medium (viewer [left arrow] medium [right arrow] lifeworld).
Even a totally transparent visual medium, however, is not a neutral transmitter of sense data. Media technologies transform visual experience in fundamental ways (Ihde 1983: 55-58). By bridging spatio-temporal horizons, media amplify a viewer's visual powers. By providing a visual experience that only approximates the lifeworld, without its full richness, media are visually reductive. By making physically inaccessible objects appear close, media lend to vision a peculiar, irreal quality. By enacting instantaneous shifts of point-of-view, media editing causes a disconnection or disjunction of ordinary experience of space-time. All of these media transformations of a viewer's engagement with the lifeworld--amplification, reduction, irreality, and space-time disjunctions--change the total perceptual situation and become part of the viewer's reality. As Ihde says, media have a "real" effect (1983: 59).
Media transformations of human experience do not detract from a viewer's narrative experience but actually enhance it. Most people do not expect media narratives to be true to their lifeworld correlates. They expect media stories to be more dramatic, stirring, moving, compressed, and conclusive. British film director Alfred Hitchcock, a master of cinematic storytelling, famously said, "What is drama, after all, but life with the dull bits cut out" (Truffaut 1983: 103). His comment can be generalized to all visual media, which amplify everyday experience of stories in the lifeworld, reduce stories to their most intense moments, make stories seem irreal, and through editing cause space-time disjunctions. Embodiment is a viewer's pre-reflective, experiential tool for perceiving a narrative flow from a disjunctive sequence of spectacles in a movie, computer game, virtual reality web site, or other media. Without embodiment, we would not be able to perceive the stories that tie together media spectacles, and screen media could not function as storytelling devices.
Embodiment of screen space
Most visual media, including print and screen media, have an overall, two-dimensional presentation plane, delimited by a fixed format or aspect (proportion of width to height), usually rectangular. Screen media place a viewer's optimum point-of-view approximately perpendicular to the center of the presentation plane. Even so-called S3D motion pictures with synthetic stereoscopic vision depend on a fixed aspect and centered viewing perspective. Curiously, viewing screen media at an oblique angle does not cause a breakdown of screen perception, if the viewer's visual field is able to embody a centered perspective (see Fig. 6). An oblique angle of view in a movie theater might produce visual impressions of tall, spindly, "Giacometti-looking" humans on the screen, but the viewer will visually interpolate the distorted shapes as normal human beings. This interpolation occurs automatically and pre-reflectively in perception.
As previously noted, the gaze of a screen medium converges with and locks in a viewer's own eye gaze, both enabling the viewer to see and limiting the viewer's visual field. The screen aspect is an inviolable outer horizon. The screen's experiential "imprisonment" of a viewer's point-of-view is a profound spatial transformation warranting screen media as a special case of mediated experience, represented by the second nested ellipse in Figure 1.
Screen format impinges on visual perception to the extent that it differs from the shape of the normal human visual field, which is a fuzzy-edged, horizontal ellipse of approximately 3.5:1 aspect ratio for two-eyed people, including the author and a number of his students (see Fig. 7).
Screen media formats are strikingly narrower than the human visual field, causing a prevailing consciousness of, even a fascination with, the left and right frame edges. Through perception of figure-ground relationships, a viewer perceives screen space as extending beyond the frame edge. Because the Earth's gravity coerces a horizontal orientation of movement and action, humans pay much more attention to the left-right dimension of the media frame than to up-down dimension. Zettl's term for this off-screen, mental "completion" of viewed objects is psychological closure (2010: 116), though the phenomenon is not, in fact, a psychological thought process, but rather, a structure of visual perception (Stroker 1987: 124).
The philosopher's expression for the phenomenon of seeing beyond the frame edge is "seeing with ..." or apperception (Husserl 1962: sec. 53, 149-151). The net effect of apperception is horizontal stretching of screen space (see Fig. 8).
Compared to direct experience, screen space appears stretched out--the narrower the screen format, the stronger the stretching effect. This counterintuitive effect has been noted repeatedly by visitors to broadcast television stations, who invariably comment that the studio set appears smaller to them than what they perceive when they watch the show at home. Their televisual perception seems wider because the TV frame restricts the horizontal field of view, causing them to apperceive a wider coextensive screen space, while the studio set might in fact be crammed into a small physical space.
In rare viewing situations, a screen encompasses a viewer's entire peripheral vision. Such situations might include an extremely close viewing position, a very large screen such as IMAX cinema (see Fig. 9), or multi-projection environments (Youngblood 1970: 387-398).
Without a screen edge vignetting the visual field and stimulating apperception, little or no horizontal stretching occurs, but the screen image still compels a viewer's embodied perception of changes in point-of-view.
In summary, a viewer perceives screen media as horizontally stretched as compared to normal experience. The next section explains how the perceived horizontal elongation of screen space impacts a viewer's experience of time and narrative.
Embodied space-time and narrative
Everyday life suggests what theoretical physics has proven: space and time are woven into an indivisible entity, space-time. Consequently, media transformations of spatial experience necessarily must impact the experience of time as well, and vice versa. Before a story scene is set, before characters' motivations are developed, before plot is established and conflict is brought to climax, a viewer's perceptual foundation for storytelling is seriality, or sequencing of events. For a story to be perceived, one thing must lead to another. This section presents a foundational theory of embodied space-time in screen media that has major implications for storytelling in digital media.
A fundamental problem for media storytellers is the lack of concrete visual evidence for abstract time (Stroker 1987: 121). A viewer only and always sees here-now. The ancient Greek philosopher Xeno of Elea went so far as to suggest that all apparent motion can be broken down into frozen moments (Zettl 2010: 270). The traditional storytelling tricks for cueing the passage of time are metaphorical, not direct evidence. Devices such as showing the hands of a clock, dissolving from one location to another, voiceover narration, or aging effects require a viewer to reflect on memories, which are re-presentations of the past brought into the present moment. While such tricks can be useful to the storyteller, they cannot make viewers see and feel narrative.
According to Edmund Husserl, pre-reflective time consciousness is a latent sense of duration behind every perception of here-now. The present moment has a kind of apperception of its own prior beginning and not yet completed future (1962: secs. 81-82, 215-219). The precise Husserlian phenomenological mechanism for this past-present-future perceptual process is beyond the scope of this discussion. It is sufficient here to note that a viewer's only insight into the flow of time in mediated experience is a changing visual image of an enduring thing in space. A rapid change is able to show the flow of time more clearly than a slow one (Stroker 1987: 38). In screen media, such temporally revealing changes are shown either by movements of the subject or movements of point-of-view, or both; in either case, a viewer's time consciousness depends on seeing a subject as remaining the same spatial thing, while the visual image of it changes. For example, a cyclist working out to the Beach Road, Melbourne video is aware of how long it takes to climb the next hill.
The serial sequencing of screen events necessary for the perception of narrative is made visible through distinct perspectival changes in a viewer's embodiment of screen space. In a media producer's vernacular, this type of presentation is known as continuity editing, which implies a perceived flow of action from one screen embodiment to the next, with the embodiments of visual fields or "shots" having distinctly different angles of view on the same subject (see Fig. 10).
In other words, the replacement of one embodied point of view by another embodied view of the same subject is a viewer's best perceptual evidence of the passage of time in screen media. Though a continuity edit takes zero time to occur, it enables a viewer to perceive the passage of time across the two shots that are joined by the edit.
A brief accounting of the two extreme counter-cases to continuity editing show that they do not give perception of time's passing: First, other types of screen media editing, such as montage, do not provide direct perceptual evidence of the passage of time because of the lack of overlap in embodiments; the various screen events edited together could, in fact, be simultaneous. A paradigmatic example of media montage is the Odessa Steps sequence in the feature film Battleship Potemkin (1925) by Russian director Sergei Eisenstein. A more recent example of montage is the title sequence of the feature film Persona (1966) by Swedish director Ingmar Bergman. In both examples, widely divergent things flash on the screen, with no overlap. According to Elisabeth Stroker, non-continuity edits deny consciousness its time awareness by short-circuiting changing visual perspectives: "The identity of the sense guarantees the relationship to a now of something that is passing and makes comprehensible the 'now' and the 'having been'" (1987: 39). Second, an unedited screen image also is inadequate perceptually for time awareness. It is tempting to suppose that a long, continuous screen image with no breaks, such as the aforementioned cycling workout video, in which screen time is identical to the viewer's time, would provide truly faithful temporal experience. However, numerous film and video experiments have shown a gradual decay of time awareness owing to the irreality of sustained views of the same embodied screen perspective. Screen media examples with long takes include the feature film Rope (1948) by British director Alfred Hitchcock, the minimalist experimental film Sleep (1963) by U.S. artist Andy Warhol, and the minimalist video I Do Not Know What It Is That I Am Like (1986) by United States artist Bill Viola. Because of its lack of edits, the cycling workout video has a superimposed clock to reinforce the passage of time.
If perceiving narrative depends on visualizing seriality, and if continuity edits are the audience's best perceptual evidence of the passage of time in screen media, then the appropriate conclusion is that continuity editing most clearly visualizes stories in screen media. This conclusion has intuitive appeal. Continuity editing dominates commercial film and television storytelling. Indeed, the locking in of viewers' perspectives with the screen necessitates threading together different coerced perspectives in the continuity editing process.
What if mediated experience were released from the imprisonment of the screen, as in virtual media? Computer games and virtual reality web sites currently are screen-bound, but virtual media of the future will be projected holographically in unbounded, three-dimensional space (see overlapping ellipses for screen media and virtual media in Fig. 1). If imprisoned visual perspectives and editing are the most nontransparent aspects of screen mediated experiences as compared to direct experience, then holographic virtual media promise to transform time and narrative by liberating the viewer to engage freely in mediated space-time. Phenomenological analysis must wait for this new technology to emerge. In the meantime, virtual media such as computer games and virtual reality web sites remain bound by the screen's perceptual rules.
Continuity editing in screen media condenses experienced time. This time condensation accounts for the compelling nature of screen narrative. This temporal transformation in screen media shall be known euphemistically as time leakage.
Time leakage is a consequence of the horizontal stretching of screen space, caused by apperception of a coextensive lifeworld beyond the frame edge. As previously noted, this stretching effect is stronger in narrower screen formats, so time leakage also will be greater for these formats. The proof of this claim is found in relativity: when space is expanded, time must move faster to maintain the unity of spacetime. For example, travelling 100 miles in two hours requires a speed of 50 miles per hour; to travel twice the distance (200 miles) in the same amount of time (two hours) requires doubling the speed to 100 miles per hour. In the same way, stretched screen space is compensated by accelerated time, perceptually speaking, though actual elapsed time is unchanged. What is the explanation for this irreal quality of screen space-time that stretches and accelerates? The answer to this puzzle lies with the viewer's embodiment of screen perspectives facilitated by continuity editing.
Watching a screen story is not a passive experience; on the contrary, it is demanding and engaging perceptual work. Each continuity edit demands a viewer's embodiment of a new observing position in screen space (see Fig. 10). The perspectival shifts expend considerable amounts of psychic energy. The cumulative psychic energy toll of a rapid-fire continuity sequence can be enormous. Yet, all this energy expenditure for embodying the continuity edits occurs in zero time. How can such high energy interactivity occur instantaneously? The only possible resolution of this embodied time contradiction is that the compressed time of the narrative screen experience leaks into the continuity edits. Zettl observed that the more frequent the edits, the greater the time compression (2010: 241-242).
Playing screen-based computer games or navigating virtual worlds on the Internet hint at the possibility of holographic virtual media. Holographic visual experience would not be confined by a screen frame and therefore would not entail horizontally stretched space or time leakage. Edits presumably would be chosen by viewers and afforded time for embodying the new perspectives. By shedding its screen "blinders", holographic mediated experience would return the human visual field to a more transparent space-time.
Summary and conclusions
Phenomenology is suited to the job of describing mass media transformations of storytelling experience. This phenomenological inquiry specifically investigated visual perception of narrative. Screen media transform the visual experience of narrative in fundamentally important ways. Humans make sense out of the world by projecting their visual points-of-view in space by a pre-reflective process known as embodiment. Screen media viewers use embodiment to situate themselves in storytelling space according to screen requirements. A unique consequence of embodying images limited by the frame of the screen is apperceiving a coextensive world that extends beyond the frame edge. This seeing with ... a larger lifeworld beyond the frame causes a horizontal stretching of experienced screen space. When coupled with continuity-style editing, which most clearly conveys serial changes in perspective needed for narrative, the horizontal stretching of screen space causes a corresponding time compression and event intensification. This media transformation is represented as time leaking into the zero-time transitions of continuity editing.
Virtual media currently obey the laws of the screen, but holographic virtual media of the future have the potential to free the viewer from the imprisoning screen image and its spatial and temporal transformations of mediate narrative experience. This new, less transformative storytelling mode promises to be more transparent and truer to life.
Arnheim, R. 1974. Art and Visual Perception: A Psychology of the Creative Eye. Berkeley, Los Angeles, London: University of California Press.
CTXC Training Videos. 2013. 45 Minute Indoor Trainer Workout [online], [cited 27 July 2016]. Available from Internet: https://youtu.be/zSFYRiqodvU
Heilig, M. 2001. The Cinema of the Future, in Packer, R.; Jordan, K. (Eds.). Multimedia: From Wagner to Virtual Reality. New York, London: W. W. Norton & Company, 239-251.
Husserl, E. 1962. Ideas: General Introduction to Pure Phenomenology. New York: Collier Books.
Ihde, D. 1983. Existential Technics. Albany, NY: State University of New York Press.
Ihde, D. 1979. Technics and Praxis: A Philosophy of Technology. Cohen, R. S.; Wartofsky, M. W. (Eds.). Series: Boston Studies in the Philosophy of Science. Vol. 24. Dordrecht, Boston, London: D. Reidel Publishing Company.
Jay, M. 1993. Downcast Eyes: The Denigration of Vision in Twentieth-Century French Thought. Berkeley, Los Angeles, London: University of California Press.
Merleau-Ponty, M. 2002. Phenomenology of Perception. Series: Routledge Classics. London and New York: Routledge.
Moholy-Nagy, L. 1967. Painting, Photography, Film. Cambridge, MA: The MIT Press.
Ong, S. J. W. 1958. Ramus, Method, and the Decay of Dialogue: From the Art of Discourse to the Art of Reason. Cambridge, MA: Harvard University Press.
Pilotta, J. J.; Mickunas, A. 1990. Science of Communication: Its Phenomenological Foundation. Series: General Communication Theory and Methodology. Hillsdale, NJ: Lawrence Erlbaum Associates.
Scholz, P. 2008. Personal Interview with Russell J. Cook. Loyola University Maryland, Baltimore, Maryland. Unpublished source.
Stroker, E. 1987. Investigations in Philosophy of Space. Series: Series in Continental Thought. Bk. 11. Athens: Ohio University Press.
Truffaut, F. 1983. Hitchcock. New York: Simon & Schuster.
Weber, S. 1996. Mass Mediauras: Form, Technics, Media. Cholodenko, A. (Ed.). Stanford, CA: Stanford University Press.
Wilson Nightingale, A. 2004. Spectacles of Truth in Classical Greek Philosophy: Theoria in Its Cultural Context. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511482564
Youngblood, G. 1970. Expanded Cinema. New York: P. Dutton and Co., Inc.
Zettl, H. 2010. Sight, Sound, Motion: Applied Media Aesthetics. Series: Wadsworth Series in Broadcast and Production. Belmont, CA: Wadsworth.
Russell J. COOK
Communication Department, Loyola College, Loyola University Maryland, Baltimore, MD 21210, United States
Received 5 August 2016; accepted 9 October 2016
(1) Though the other human senses are not included in this discussion of technological embodiment of media, they are part of a screen spectator's experience. Sound is more important in media storytelling than is generally supposed, and the other human senses certainly come into play through synaesthetic extension. For example, seeing and touching surface textures are closely linked (Stroker 1987: 126).
Please Note: Illustration(s) are not available due to copyright restrictions.
Caption: Fig. 1. Realms of embodiment in human experience (source: created by author)
Caption: Fig. 2. Embodiment of an imaginary topological perspective on a room (source: created by author)
Caption: Fig. 3. Embodiment of an implied perspective in a street map (source: created by author)
Caption: Fig. 4. Z-axis articulation of screen space by perceived eye gaze (source: created by author)
Caption: Fig. 5. Viewers "borrow" the eye gaze of the screen (source: created by author)
Caption: Fig. 6. Interpolation of oblique point-of-view in movie theater (source: created by author)
Caption: Fig. 7. Media aspect ratios compared to the normal human visual field (source: created by author)
Caption: Fig. 8. Horizontal stretching of screen space through apperception of coextensive lifeworld (source: created by author)
Caption: Fig. 9. The world's largest media screen: the IMAX movie theater at Darling Harbour, Sydney, Australia (source: created by author)
Caption: Fig. 10. Embodiment of various screen perspectives in continuity editing (source: created by author)
|Printer friendly Cite/link Email Feedback|
|Author:||Cook, Russell J.|
|Publication:||COACTIVITY: Philosophy, Communication|
|Date:||Mar 1, 2017|
|Previous Article:||REVIEW OF THE MONOGRAPH.|
|Next Article:||RESISTANCE TO WESTERN POPULAR AND POP-CULTURE IN INDIA/PASIPRIESINIMAS POPULIARIAJAI VAKARU IR POPKULTURAI INDIJOJE.|