Printer Friendly

Audio description on the thought-action continuum.


The term audio description refers to the spoken commentary that is added to film soundtracks to enable visually impaired people to follow the action on the screen by supplementing the information that is already available from character dialogue and other parts of the soundtrack. As this essay explains, there is currently a debate within this field regarding the possibility and desirability of "objectivity" in audio descriptions. Although this debate may appear to be of limited interest to anyone who is not directly concerned with the production and reception of audio descriptions, we feel that it raises large and important issues that are of relevance to all those who work in the much wider fields of narrative theory, stylistics and literary theory generally. We have in mind, in particular, our questioning of the apparently obvious distinction between descriptions of characters' actions and descriptions of characters' mental processes in fictional narratives.

Specifically, this essay seeks to show how concepts from narratology and techniques from corpus linguistics can be applied to the analysis of audio description. We are concerned with the issue of what should be described in audio description for feature films, and how it should be worded. We introduce narratological concepts that can help to better articulate this issue and to better analyze and compare examples of audio description. To complement the application of narratological concepts for a close reading of audio description samples, we show how corpus linguistics techniques can be used to learn about what is being described, and how, in a corpus of ninety-one audio description scripts. Our focus is on how an audio description utterance can, and, as it turns out, often does, describe a character's actions as depicted on-screen while simultaneously giving some information about the character's thoughts.

Central to the practice and theory of audio description is the question of what should be included in a description (Vercauteren). Concerning the issue of what to describe, and how to describe it, some recent guidelines for audio description state:

"The best audio describers objectively recount the visual aspects of an image. Subjective or qualitative judgments or comment get in the way--they constitute an interpretation on the part of the describer and are unnecessary and unwanted ... Describers must differentiate between emotion or reasoning (which requires an interpretation on the part of the observer) and the physical characteristics of emotion or reasoning (which are more concrete and allow description users to conjure their own interpretations)." (Snyder 17)

While striking an exceptionally unequivocal tone, these guidelines reflect a point of view that has been prevalent among audio description practitioners. Earlier guidance on standards for audio description provided by ITC is similar in advising not to "interpret events" or "give away the plot" (ITC). The ITC guidance also flags a contentious point about whether or not to give information "that is not apparent on the screen," without really explaining how to determine what is "apparent." It seems to us that such guidelines leave unanswered questions about what information is necessary for an audience to understand and enjoy a film, and what "interpretation" means in this context Also, and this becomes the main theme of the essay, with regards to what can be seen and hence described, we take issue with the supposedly clear-cut distinction that is made between a character's "emotion or reasoning" (thoughts, which should not be mentioned according to the guidelines) and their "apparent" physical manifestations (actions, which may be described).

This essay seeks to contribute to audio description theory and practice in two ways. First, we introduce more rigorously defined terms from narratology to clarify and simplify the issues alluded to in the previous paragraph. Second, we show how corpus linguistics techniques can be used to characterize the information about characters' actions and thoughts that audio description actually provides. We, thus, hope to help circumvent the "misguided dilemma of subjectivity versus objectivity" (Kruger and Orero 141) and further develop the multidisciplinary nature of audiovisual translation research (cf. Matamala and Orero).

Recent years have seen the uptake of various narratological concepts and techniques by audio description researchers. Through a corpus-based analysis, Salway (2007) showed how the language of audio description is influenced by its narrative function, i.e., it must convey information about "chains of events in cause-effect relationships" where "the agents of cause and effect are characters with goals, beliefs and emotions" (152). With the aim of teaching audio describers, Remael and Vercauteren (2007) show how to identify "narratively significant cues" for inclusion in descriptions in relation to the exposition phase of filmic narratives. Likewise, Ibanez (2012) argues for the importance of explicating film narrative in the teaching of audio description. An innovative approach correlated eye-tracking data with viewers' narrative constructions to understand how different visual elements contribute to storytelling (Kruger). Narratological theory was drawn on to argue for a "more subjective and cultural-historical reference point for audio describers" (Finbow 215) while Vandaele (2012) also extended consideration of narrative beyond the depicted sequences of events in films to include the hypothetical events that are triggered in the viewer's mind.

Of particular relevance for the current essay is Kruger's argument that the filmic text, shorn of the moving image, must be "re-narrativised" for blind and visually impaired audiences (Kruger 231-32). This means giving consideration to how the story told by the film can be retold by combining a new speech track with the dialogue, sound effects, and music. Regarding the accessibility of films, Kruger makes the case for giving less priority to a precise recounting of on-screen action and more priority to the "narrative implication or effect of what can be seen" (234). To achieve this, "audio narration" (233) is suggested as an alternative to audio description, where there is a continuum from objective description (that may be achieved in the audio description of a documentary) to a narration that may use, for example, "markers of focalization ... to enable the audience to situate themselves in the fictional world" (243). At the core of the current essay is the somewhat analogous and complementary thought-action continuum.

Informed by a narratological perspective, our premise is that understanding a story entails knowing about the mental states of its characters. In other words, it is essential for the audience of a film to learn about the causal network of beliefs, desires, and goals that motivate and explain the characters' actions. At first glance, this perspective may seem to be at odds with the guidelines mentioned above: can it be possible to see, and hence to describe mental states? Within the moving image of a film--for which audio description is a surrogate--actors act, and with their actions and through their facial expressions, they reveal something about characters' mental states; of course, the other components of a film (dialogue, music, and film techniques) contribute to the storytelling too.

The central question that we address here is this: how much information about the "minds," i.e., mental states, of characters in films is audio description able to convey--given the constraints of being objective (not "interpreting"), allowing audiences to make their own inferences about a story (not "giving away the plot"), and the limited time available for description (which must fit between the film's dialogue)?

In section 2, we introduce some core narratological concepts and discuss how they are relevant to audio description research and practice, especially to the issue of characters' mental states. In section 3, we describe how corpus analysis techniques can be used to analyze larger samples of audio description and develop a preliminary classification of ways in which describers convey information about characters' mental states. We close in section 4 by discussing the synergies between audiovisual translation, narratology, and corpus linguistics.


In section 2.1, we introduce some fundamental concepts from the field of narratology and discuss how these can clarify thinking about audio description with an emphasis on the thought-action continuum. In section 2.2, this is used to compare two audio descriptions of the same film in terms of how they provide information about the minds of the characters.

2.1 Why Should Audiovisual Translation Be Interested in Narratology?

Narratology (or narrative theory) is the branch of literary study that is devoted to the analysis of narrative. In its modern form, it grew out of the French Structuralist movement of the 1960s. It has, however, recently become more eclectic and inclusive in its use of the techniques used to analyze narratives (Herman). Narratology studies the nature, form, and functioning of all narratives irrespective of their mode or medium of representation. Indeed, many narratological concepts are particularly suited to multimodal analysis (Ryan). For example, narratologists distinguish between story (often referred to by the Russian term fabula) and discourse (sjuzet). Story is the content plane, the "what" of narrative, the narrated; discourse is the expression plane, the "how" of narrative, the narrating. So, the story of Cinderella can be told differently in any number of different discourses or media (a short story, a film, a play, a ballet, a cartoon, etc.) and still remain the same story. The application of narratological concepts to a multimodal activity such as audio description, which uses language to describe film, should prove to be especially fruitful. For example, audio description tends to be what narratologists call hehaviorist narrative. Behaviorist narratives are those that are limited to conveying characters' words and actions and do not give direct access to their thoughts and feelings. However, some narratologists emphasize how much information even behaviorist narratives are able to convey about characters' mental states (Palmer 206-7), and it is hoped that the discussion that follows will contribute to this debate.

Within narratology, the study of what are called fictional minds is the study of the mental functioning of the characters who inhabit the storyworlds created by fictional narratives. For example, in relation to written narratives, it addresses the question: how do we, when reading a novel, construct from the words in the text an awareness of the mental functioning of the characters in that novel? Readers enter the storyworlds created by novels and then follow the logic of the events that occur in them primarily by attempting to reconstruct the fictional minds of the characters in that storyworld. Otherwise, readers lose the plot. These constructions of the minds of fictional characters by narrators and readers are central to our understanding of how novels work, because fictional narrative is, in essence, the description of fictional mental functioning. It is not possible to follow the plot of a novel without following the thought processes of the main characters in that novel. In fact, the plot consists to a great extent of those thought processes.

As suggested by the reference above to the multimodal nature of narratology, this approach can be applied as easily to films as to novels. So, to put the issue of audio description in the context of this work in narrative theory, it could be argued that the point of audio description is to help the audience to follow the mental functioning of the characters in the film. For example, the guidance on information and objectivity can be usefully restated in terms of fictional minds. Descriptions should contain (among other things) essential information about fictional minds but should avoid both obvious and unnecessarily subjective inferences about fictional minds.

It may appear at first glance that there is a tension here between this cognitive focus on characters' minds and the guidance that audio descriptions should be as objective as possible. However, the purpose of the following discussion is to show that there is no conflict. Even the most apparently objective action descriptions often make important inferences about characters' mental functioning in order to progress the plot.

It is tempting to think of action descriptions and descriptions of consciousness as two very different things. The former describe physical movements, and so on, and the latter convey private flows of thought. But it is not as simple as that There is a continuum rather than a simple dichotomy. It is called the thought-action continuum (Palmer 212-14). Here is a simple example. "They are hiding behind the curtain" looks like an action description pure and simple. But compare it to another, similar phrase: "They are standing behind the curtain." In the context of the second example, the first description, "hiding," starts to look very different. It contains important information about the mental functioning of the people standing behind the curtain because it explains the reason why they are doing so. Saying they are "standing" there leaves open any number of reasons why they would be standing there. From this angle, the more a reader looks at the word "hiding," the more like a description of consciousness it becomes. Put another way, the word "standing" is at the action end of the thought-action continuum; the word "hiding" is nearer the middle, because it describes the action but also contains a reference to the mental functioning behind the action. It is worth mentioning that the mental states referred to in words in the middle of the continuum are often, unsurprisingly of course, reasons, motives, and intentions relating to action.

In the Philosophical Investigations (1958), the philosopher Ludwig Wittgenstein quotes the statement "I noticed that he was out of humour," and asks, "Is this a report about his behaviour or his state of mind." When "Wittgenstein's question" (see Palmer 120-21) is asked of the statement "They are standing behind the curtain," the answer is that it is a report of their behavior; when asked of the statement "They are hiding behind the curtain," the answer is that it is a report of both their behavior and their state of mind. And Wittgenstein's question can usefully be asked of many of the statements made in action descriptions.

The other large group of mental states that often occur in these statements is emotions. For example, in one of the scenes from the film The English Patient to be discussed in the following section, one of the audio descriptions refers to the female protagonist, Katherine, as simply stopping and turning toward Almasy. That is at the action end of the continuum. The other description says that "She rounds on Almasy." The second description conveys much more information about her emotions of anger and frustration. So it is in the middle of the continuum. Similarly, for example, in the same scene, "Katherine glares" at Almasy, rather than the more neutral "Katherine looks at him." A hypothetical example at the thought end of the continuum would be something like "Katherine is angry with Almasy."

2.2 Analyzing Three Scenes from The English Patient

The broad framework outlined above can now be applied to a comparison between two different audio descriptions (shown as "A" and "B" below) of three short scenes from Anthony Minghella's film, The English Patient (1996). All of these scenes occur fairly early in the film and show the beginnings of the love affair between the two main characters, Katherine and Almasy, together with the growing concern of Katherine's husband, Geoffrey Clifton. The discussion will look at both the similarities and the differences between the two audio descriptions of these three scenes.

Scene one: As part of the communal entertainments around the desert campfire, Katherine is telling the expedition party a story:

"Katherine hesitates as she meets Almasy's gaze across the campfire." (A)

"Katherine catches Almasy's eye. He is transfixed. Clifton's smile fades. Katherine smiles awkwardly." (B)

B is more informative than A, even though it is only four words longer, telling us in only thirteen words a good deal about the states of mind of the three characters. With regard to A, we are given no clue as to the various, very different reasons why Katherine might hesitate. With regard to B, it is a rewarding exercise to ask Wittgenstein's question here. Is Almasy looking transfixed (an action) or feeling transfixed (a state of consciousness) or both? It is suggested that in the case of all of the four sentences in B, the answer to the question is: both. These are reports of characters' behavior and also of their states of mind. In other words, all of the four sentences would be placed in the middle of the thought-action continuum. The audience will know that Katherine and Almasy are very awkward with each other because their relationship is growing. Each is intensely aware of the other. Equally, the description of Clifton's behavior, his smile fading, clearly indicates his state of mind. He is uneasily aware of the growing attraction between his wife and another man. However, as could be expected, the description also leaves open some intriguing areas of speculation and doubt. For example, saying that Katherine catches Almasy's eye raises issues of motives and intention. Does she intend to catch his eye, or does she do it accidentally? It is likely that listeners to the audio description will disagree on how they interpret Katherine's behavior within the context of the development of the relationship so far in the film.

Scene two: During a dance at the Cairo hotel, Katherine has just finished dancing with a member of the group:

"Almasy steps between them and takes Katherine in his arms. Seeing them, Geoffrey sips his wine thoughtfully. On the dance floor, Almasy does not answer. He holds her stiffly in his arms, gazing directly at her finely sculpted features and pale blond hair. Disconcerted, Katherine meets his gaze, then glances away uncertainly. Without speaking they glide across the floor between the dancing couples circling beneath the crystal chandeliers." (A)

"Almasy cuts in and leads Katherine. She smiles. Almasy studies her intently and she drops her gaze. When she lifts her eyes again he is still staring. He holds her in a stiff embrace, his face inches from hers, his hand resting on the small of her back pressing her close to him." (B)

In this scene, the two descriptions are quite similar in some ways: both emphasize Almasy's taking control of the situation; both use the word "stiff" to describe Almasy's dancing; and both have Katherine glancing away. However, there are some differences. Version A makes more of Katherine's turning away movement, using the words "disconcerted" and "uncertainly" to provide important information about her mental states. Also, it is odd that, as in the last scene, only one description takes note of Clifton's reaction to the relationship between Katherine and Almasy. But this time, it is A rather than B.

In addition, the difference in what may be called focus is interesting. B emphasizes the almost claustrophobic intimacy of the physical contact between the pair: his face inches from hers, pressing her close. It conveys the tension and the growing attraction between them. By contrast, A stresses the social and physical context of the dancing couples and the crystal chandeliers. The use of the word "glide" suggests a much more relaxed encounter. Very different mental states are implied: it makes the two characters' states of mind sound misleadingly relaxed and uninvolved by comparison.

Scene three: Back in the desert, Katherine shows Almasy some of her paintings: "She offers them to him. He takes the paintings and looks at them. He holds them out. She takes the pictures back. Katherine walks a few steps then stops and turns. He looks embarrassed. Quickly she walks away. Almasy looks after her uncertainly." (A)

"Almasy takes them and glances up at her. Katherine smiles as he looks through them. He hands them back. Katherine's face falls. Reluctantly she takes the paintings and starts to go. Then hesitates. She rounds on Almasy. He opens his mouth to speak, but says nothing. Katherine glares, turns and marches off, heading up the side of the dune. Almasy blinks, sighs and shakes his head. He returns to his book." (B)

In this scene, B is noticeably more involved in Katherine's emotions than A. In narratological terms, the scene is focalized through her. It is presented from her point of view. The words used are more vivid: she "smiles," she "hesitates," she takes the paintings "reluctantly." She "rounds" on Almasy, she "glares," "turns and marches off." Much of this is veering toward the thought end of the thought-action continuum. Her conflicted thought processes are made clear. By contrast, A stays at the action end of the continuum in relation to Katherine. Significantly though, the A focus is on Almasy's mental functioning: he "looks embarrassed"; he looks after her "uncertainly."

We did not notice any consistent patterns of difference between the two descriptions across the three scenes. For example, Clifton's reactions were noted in description B for scene 1 and in description A in scene 2. So there were no obvious patterns of identification, say, with one character in one description and another character in another. It may be that this point could form the basis of at least one good criterion for objectivity, i.e., descriptions should not over-identify with, or get over-involved in, one character at the expense of the others. What we did notice in both descriptions were some common devices used to give information about mental states. These include descriptions of different types of smile ("smile fades," "smiles awkwardly") and different ways of looking at other characters ("studies her intently," "looks after her uncertainly," and descriptions of characters appearing to be in a particular state ("he is transfixed," "disconcerted," "looks embarrassed"). These devices become the starting point for our corpus analysis in section 3.

Given that the two descriptions are so informative about the three main characters' states of mind, the question arises as to whether they meet the guidelines' requirement to be objective. Our close reading of some audio description fragments leads us to claim that it is possible to remain objective while being informative about mental states in a concise and consistent way, and that such information is indeed essential for the audience to understand the story. To put the point in narratological terms: the descriptions are in appropriate places on the thought-action continuum. We have also shown how small differences in descriptions of characters' actions can result in big potential differences for an audience's understanding of a story, which is perhaps something to be considered in future guidelines.


In this section, we seek to demonstrate the utility of corpus analysis techniques for surveying common words and phrases, and hence, the common kinds of information conveyed by audio description. In particular, we will use these techniques to gauge the extent to which our samples of audio description provides information about characters' mental states, and to develop an account of the options audio describers have for conveying such information. Automated text and corpus analysis techniques are used in many fields, for many purposes, and a review of these is beyond the scope of this chapter--for an introduction see Biber, Conrad, and Reppen (1998). In broad terms though, individual texts can be compared according to the frequencies of words and phrases occurring in them, and the co-texts in which these words are used. In section 3.1, we use the "Word List" and "Concordance" functionalities of the freely available corpus analysis tool AntConc (Anthony) to extend the previous comparison of two audio description scripts. In section 3.2, we apply these techniques, and others, to a corpus comprising British English audio description scripts for ninety-one films.

3.1 Further Comparison of Descriptions A and B

The detailed analysis that follows may be summarized in this way. Some of the ways in which audio describers can give information about characters' mental states are listed here roughly in order from action-oriented to thought-oriented. This order implies an increase in the extent to which the describer makes explicit for the AD audience what the sighted audience is expected to infer from the moving image.

(1) Description of simple actions that are intended to imply mental states, e.g., "Robin rolls his eyes."

(2) Description of facial expressions, such as "smiles," "frowns," and "grins."

(3) Modification of the description of actions by adding an adverb, e.g., "walks cautiously," by using a troponym, e.g., "creeps," or with a phrase like "smiling in relief." The particular choice of adverb, troponym, or abstract noun will lead to a different positioning on the thought-action continuum.

(4) Description of a character as appearing to be in a particular mental state, e.g., "Harry looks confused."

This is, at best, a preliminary attempt at a categorization. It does not include actions such as "hide" (f=60), most likely because they do not take a distinctive form or co-text in audio description. This suggests that our data-driven approach may need to be complemented by an approach that starts with a semantic classification of verbs listing verbs such as "hide" that fall midway on the thought-action continuum. However, what is certain at this stage is that British audio description is rich in descriptions of characters' minds--including information about their motivations, emotions, and feelings toward others--that are described in a relatively constrained variety of forms, and that there seems to a be a tendency toward descriptions around the midway point of the thought-action continuum (summarized as point 3 above). What we tend not to see examples of are "pure thought" descriptions, such as "Harry feels confused."

Let us now look at a detailed comparison of two audio description scripts for three scenes of one film (section 2.2). Certain words stood out for the ways in which they informed about characters' mental states: the word smiles, especially when followed by an adverb; the word looks followed by an adjective or an adverb; and, words describing ways of looking such as gazes and glances. How are these and other words used throughout the two descriptions to convey information about characters' minds?

A frequency count showed that in the whole of description A, the word form "smiles" occurred forty times, and in B it occurred fourteen times. (1) In eight instances in A, "smiles" was followed by an adverb that seems to give more insight into a character's mental state than "smiles" alone; in B there were two instances. All ten examples are listed below.
   She smiles benignly.
   She smiles fondly at the man with the melted face.
   She smiles fondly at him.
   Katherine smiles awkwardly.
   She smiles fondly and looks over at Almasy.
   He smiles briefly at her in the darkness.
   She smiles weakly.
   He smiles ruefully and lowers his head, looking pensive.
   Almasy smiles faintly.
   Lady Hammond smiles delightedly as Katharine crosses the courtyard.

It is unsurprising that the writers of audio descriptions rely on stock or standard devices that enable them to convey information about characters' mental states. These frequently recurring terms may be thought of as "pegs" on which intention-rich adverbs can be hung. It seems that the "smiles" peg is congenial to A but less so to B. Other favorite pegs include "looks" and "walks" and these are discussed below. It is easy to see why "smiles" and "looks" should be popular pegs; after all, the face can be very expressive of internal mental states. The examples listed above are all in the middle of the thought-action continuum because they are descriptions of actions that convey states of mind. Perhaps "briefly" and "faintly" are nearer the action end of the spectrum than the others because they require a little more context to make the indicated state of mind clear, e.g., musical cues or other filmic devices.

The word "looks" occurs fifty-six times in description A and thirty-one times in B. It is used in these audio descriptions in two main senses: to look at someone or something, and to appear to be in a certain mental state. It is used in the latter sense seven times in both descriptions A and B. The use of "looks" in the latter sense (appearing to be in a particular mental state) is close to the thought end of the spectrum. This is because there are not really any actions being described. It is a much more direct attempt than the former sense to convey mental states. "He looks sad" could easily be reformulated as "he appears to be sad." This latter formulation would make it clear that there is no action involved. In the following examples of the latter usage, the apparent state of mind is very clearly indicated.
   Almasy looks thoughtful.
   He looks uncertain.
   Katherine is watching but looks troubled.
   In the cockpit Clifton looks grimly determined.
   The patient looks drained.
   She looks happy for him.
   Then Hana looks grave again.
   Hana looks concerned.
   Madox looks puzzled.
   He looks puzzled.
   Almasy looks annoyed.
   He looks embarrassed.
   He looks upset and agitated.
   She looks unhappy and tearful.

The uses in the former sense (looking at something) tend to be nearer the middle of the spectrum because there is an action involved. In both descriptions A and B "looks" is modified by an adverb seven times to give more insight into mental states. It is clear that, even in the action uses of "looks," there is plenty of information being given about mental states. Indeed, it is notable that several words convey the presence of a good deal of highly self-conscious thought: "thoughtfully," "pensively," "uncertainly," "curiously," and "questioningly." This is not surprising given the subject matter of this film--highly intelligent and sensitive people falling in love.
   He looks thoughtfully from the hand to the surrounding rocks.
   ... he looks thoughtfully in the direction Katherine has taken.
   He looks pointedly at Katherine.
   Hana looks after him forlornly.
   He sets the syringe to one side and looks directly at the patient.
   He looks about him furtively.
   He looks gravely down at her, unsure.
   She looks pensively at the bell-tower of the monastery.
   She heaves a dusty mattress on-to the bed and looks around
   He looks at her awkwardly.
   Almasy looks after her uncertainly.
   ... and looks at him curiously as he digs in the sand.
   She looks at him questioningly.
   Hana touches his arm and looks at him fondly.

Of course, aside from smiling, looking, and walking, there are many other actions that could be modified by adverbs to inform about mental states in audio description. In order to focus on how these actions are performed, the next step was to search the two descriptions for words taking a common and easily identifiable form of English adverbs, i.e., those ending in -ly. From inspecting the concordances of -ly adverbs, we found that in description A over 100 of the 149 instances convey information about characters' mental states; in B it is over 90 of 136 occurrences.

As an alternative to modifying "looks" with an adverb, audio describers working in some languages, such as English, have a variety of troponyms at their disposal with which they can specify a particular manner of looking, and in so doing give more insight into characters' mental states, e.g., "gazes" and "glares." We sourced a list of troponyms of "looks" from the online lexical database (Fellbaum) to see how frequently these words occurred in descriptions A and B (see Table 1); their concordances were inspected to check that they were being used in the sense of a troponym of "looks."

A uses a total of forty-seven of these troponyms, whereas B uses only fifteen. The choice of words in this case does not make much difference to the place of a statement on the continuum. The main difference appears to be the degree of attention ("stares" denotes more attention than "glances") but the word "stare" in itself does not convey the reason for the attention. For this, the context is required. However, the choice of "glares" over the basic "looks" does suggest a feeling of anger from one character toward another. We also counted the frequencies of troponyms of "walk." The following, which may be indicative of mental states as well as physical condition, were found at least once in one or both of A and B: "creeps," "hobbles," "staggers," "steps," "strides," "stumbles," "swaggers," "toddles," "totters," "waddles."

The data presented above allows us to say something about the extent to which each description conveys information about characters' minds: overall, description A contains many more of the words that we take to be related to characters' minds than description B. Our method used relatively simple corpus analysis techniques to good effect, but it is important to note that the need in places for manual inspection of concordances means that it will be impractical to replicate parts of this analysis on a much larger sample of audio description. Another problematic issue to be aware of is that the analysis was led by our prior assumptions about what words are used to convey information about characters' minds. While our results support the idea that "smiles," "looks," their troponyms, and -ly adverbs are indeed important devices for audio description, (2) we must recognize that there may be other important devices that we have not accounted for.

3.2 Information about Mental States in a Corpus of Audio Description Scripts

It could be argued that the prevalence of information about characters' minds observed above was due to the choice of film, which is a drama intimately concerned with the mental states of its protagonists. To understand more generally how audio description conveys information about characters' mental states, and to see how we can scale-up our analysis, we now turn to a corpus of ninety-one British English audio description scripts compiled to be representative of major film genres. For more details about the corpus and a corpus-based investigation into audio description as a "special language" see Salway (2007). The analysis here is based on the distribution of word forms and the co-texts in which they occur. We start by looking at -ly adverbs, at their usage with "looks" and "walks" and at troponyms for these verbs. We then extend our search for words germane to the current analysis in three ways: (i) by automatically generating lists of words using co-textual cues such as "looks -ed" and "smiling with ... (ii) by using WordNet as previously; and (iii) by drawing on results from an earlier analysis of this corpus. For the most part, the analysis was carried out using the previously mentioned "Word List" and "Concordance" functionalities. However, to easily extract lists of words occurring in particular contexts, some programs were written in the Perl programming language (3) which is optimized for text processing with regular expressions. (4) The overall impression that we get from the analysis reported below is that information about characters' minds is generally widespread in the sample of audio description available to us.

In our audio description corpus, the top twenty -ly words, with their frequencies, are "slowly" (f =439), "gently" (f =190), "quickly" "nervously" (f =125), "suddenly" (f =125), "carefully" (f =114), "thoughtfully" (f =78), "anxiously" (f =70), "intently" (f =69), "desperately" (f =55), "happily" (f =52), "sadly" (f =50), "awkwardly" (f =46), "frantically" (f =42), "cautiously" (f =40), "tenderly" (f =40), "wearily" (f =37), "curiously" (f =36), "angrily" (f =32), "warily" (f =32). These alone, in addition to a long tail of less frequent words, total 1,874 instances which, make over twenty instances per film on average. Let us now consider how two commonly described actions--looking and walking--are modified with -ly words (see Tables 2a and 2b). It seems that each action has a preferred set of adverbs, most (if not all) of which add information about mental states.

As noted in section 3.1, troponyms may be used as a compact way to convey action and thought simultaneously. For "looks" we counted the following: "stares" (f =659), "gazes" (f =180), "peers" (f =178), "glares" (f =66), "studies" (f =49), "considers" (f =27), "peeps" (f =15), "squints" (f =15), "gapes" (f =14), "regards" (f =13), "gawps" (f =12), "peeks" (f =11), "admires" (f =7), "leers" (f =4), "eyeballs" (f =3). Troponyms for "walks" included the following: "steps" (f =711), "strides" (f =162), "hurries" (f =137), "rushes" (f =122), "wanders" (f =90), "marches" (f =84), "strolls" (f =6i), "creeps" (f =54), "paces" (f =52), "staggers" (f =48), "glides" (f =45), "stumbles" (f =42), "clambers" (f =35). These troponyms, from "steps" to "admires," fall across the thought-action continuum.

We have seen that "looks" can also be used in a different sense to describe how a character appears to be in a particular mental state, often using an adjective ending in -ed, another easily identifiable form. Table 3 shows a variety of words that are used in this way. Of course, these words can be used in other contexts and when we count them in the whole corpus we find the most frequent to be "shocked" (f =53), "confused" (f =41), "puzzled" (f =40), "worried" (f =34), "stunned" (f =34), "terrified" (f =33), "concerned" (f =32), "troubled" (f =29), "surprised" (f =25). Looking at an automatically generated list of other words (not ending in -ed) that occur to the right of "looks" gives the following with their frequencies counted in the whole corpus: "blank" (f =39), "nervous" (f =30), "awkward" (f =24), "uneasy" (f =24), "thoughtful" (f =24), and many less frequent words.

The word "smiles" was noted as one of the common actions modified with -ly words. Consulting WordNet gave a list of other facial expressions; a similar list could have perhaps been derived from looking at the left context of-ly words. These were counted in the forms -[empty set], -s, and -ing ("smile," "smiles," "smiling") giving counts of "smile" (f =1,209), "frown" (f =199), "grin" (f =189), "scowl" (f =29), "smirk" (f =26), "squint" (f =25), "snarl" (f =21), "wince" (f =21), "grimace" (f =19), "glower" f =16), "sneer" f =12), "pout" (f =2), "simper" (f =1).

Looking at some of these words in context highlighted phrases such as "smiling with" and "frowning with" are followed by a word referring to a mental state. Words occurring to the right of "with" include "concern" (f =21), "relief" (f =17), "interest" (f =14), "delight" (f =14), "surprise" (f =12), "horror" (f =11), "fear" (f =10), "shock" (f =10), "alarm" (f =9), "satisfaction" (f =7), "pleasure" (f =6), "anticipation" (f =5), "worry" (f =5), "emotion" (f =5), "sadness" (f =5). In turn, inspection of these words in other co-texts pointed to the pattern "Action in Mental State," e.g., "stares in amazement." Words (abstract nouns) following "in" include "thought" (f =32), "amazement" (f =26), "shock" (f =22), "disbelief" (f =20), "horror" (f =20), "alarm" (f =18), "wonder" (f =18), "surprise" (f =14), "frustration" (f =13), "disgust" (f =10), "dismay" (f =9), "confusion" (f =9), "fear" (f =6), "awe" (f =6). Some of these words also appear following "look of" (f =44). All these words provide valuable information to the listener without access to the on-screen images about characters' mental states.

References to body parts can also be used to indicate states of mind. It was noted in Salway (2007) that words denoting body parts were unusually frequent in the audio-description corpus. Looking in more detail at how these words are used, we find the following phrases describing actions involving eyes.

closes his/her eyes (f =86)

opens his/her eyes (f =40)

eyes widen (f =24)

lowers his/her eyes (f =18)

rolls his/her eyes (f =15)

lifts his/her eyes (f =15)

narrows his/her eyes (f =14)

eyes fill with tears (f =12)

raises his/her eyes (f =8)

eyes glisten (f =8)

eyes flicker (f =8)

eyes meet (f =8)

Less frequent, but perhaps also important in the depiction of characters' minds, are actions with other body parts, including the following: "bows his/ her head" (f =16) and "hangs his/her head" (f =14), and "puts his/her hand" (f =39) and "puts his/her arm around" (f =12). Some actions are directly associated with an emotion: "kiss" (f =235), "crying" and "cries" (f =26), "weeps" and "weeping" (f =21). More rarely, some audio descriptions refer to a character's expression, and describe it as a "pained expression" (f =6), "serious expression" (f =5), or "puzzled expression" (f =4). There are also a few instances of "expression" followed by a description of a change: "expression changes" (f =10), "expression softens" (f =4), "expression becomes" (f =4), "expression darkens" (f =3).


We hope that we have shown how our twin-headed interdisciplinary approach can contribute in a number of ways to audio description research and practice, and that we have encouraged readers to explore how they may use narratology and corpus linguistics for themselves. Here we explained how the thought-action continuum and other narratological concepts can resolve apparent tensions between only describing what can be seen and the need for audiences to learn about characters' mental states, and how more generally they can help to articulate discussions about what needs to be described. This points to one potential way in which narratology could input into the revision of audio description guidelines. We also showed how corpus linguistics techniques can be used to gain a broad and empirically grounded overview of audio description practice. Maybe in the future such techniques could be used to monitor the observance or otherwise of guidelines by analyzing samples of audio descriptions as they are produced. In our research, these techniques allowed us to establish that the provision of information about characters' mental states is widespread in the selected sample of British audio description scripts, which post hoc justifies our chosen focus.

Of course, our findings do not necessarily apply to audio description in other countries and/or languages. Analyses of multilingual corpora would help to determine whether linguistic affordances and cultural preferences affect where audio descriptions lie on the thought-action continuum (for an example of multilingual research see Mazur and Chmiel). Further possible studies include investigating how the positioning of an audio description utterance on the thought-action continuum is affected by other sources of information about a character's mental state that are available to listeners, e.g., from dialogue, and from cues in the soundtrack. With respect to further synergies between audiovisual translation, narratology, and corpus linguistics, we note the potential to work in analogous ways with corpora of subtitles, and ongoing research that is looking at how patterns induced from a corpus can elucidate some of the narrative functioning of audio description in a data-driven manner (Salway).

Alan Palmer


Andrew Salway



(1.) Unless specified otherwise, all frequency counts here relate to specific word forms, rather than lemmas. We think it is appropriate to use frequency here, rather than relative frequency, because we are comparing descriptions of the same film.

(2.) Indeed, the ITC guidelines (ITC 2000) suggest the use of adverbs (point 3.9) and the use of "specific" verbs (point 3.11).


(4.) Expressions that can specify word forms and patterns to be searched for in a corpus; for more examples, see


Anthony, Laurence. AntConc (Version 3.4.2) [Computer Software]. Tokyo, Japan: Waseda University, 2011. Web. 24 Feb. 2015.

Biber, Douglas, Conrad, Susan, and Reppen, Randi. Corpus Linguistics: Investigating Language Structure and Use. Cambridge: Cambridge UP, 1998. Print.

Fellbaum, Christiane, ed. WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press, 1998. Print.

Finbow, Steve. "The state of audio description in the UK--from description to narration." Perspectives: Studies in Translatology 20.1 (2010): 215-29. Print.

Herman, David, ed. The Cambridge Companion to Narrative. Cambridge: Cambridge UP, 2007. Print.

Ibanez, Anna. "Evaluation criteria and film narrative. A frame to teaching relevance in audio description." Perspectives: Studies in Translatology 18.3 (2010): 143-53. Print.

ITC. "ITC Guidance on Standards for Audio Description." 2009. Web. 10 Aug. 2013. http:// description/index.asp.html.

Kruger, Jan-Louis. "Audio narration: re-narrativising film." Perspectives: Studies in Translatology, 18(3) (2010): 231-49. Print.

--. "Making meaning in AVT: eye tracking and viewer construction of narrative." Perspectives: Studies in Translatology 20.1 (2012): 67-86. Print.

Kruger, Jan-Louis and Orero, Pilar. "Audio description, audio narration--a new era in AVT." Perspectives: Studies in Translatology 18.3 (2010): 141-42. Print.

Matamala, Anna and Orero, Pilar. "Audiovisual translation. When modalities merge." Perspectives: Studies in Translatology 21.1 (2013): 2-4. Print.

Mazur, Iwona and Chmiel, Agnieszka. "Towards common European audio description guidelines: results of the Pear Tree Project." Perspectives: Studies in Translatology 20.1 (2012): 5-23. Print.

Palmer, Alan. Fictional Minds. Lincoln and London: U of Nebraska P, 2004. Print.

Remael, Aline and Vercauteren, Gert. "Audio describing the exposition phase of films. Teaching students what to choose." Trans. Revista de Traductologia 11 (2007): 73-93. Print.

Ryan, Marie-Laure, ed. Narrative Across Media: The Languages of Storytelling. Lincoln: U of Nebraska P, 2004. Print.

Salway, Andrew. "A Corpus-based Analysis of Audio Description." Media for All: Subtitling for the Deaf Audio Description and Sign Language. Eds. Jorge Diaz Cintas, Pilar Orero, and Aline Remael. Amsterdam: Rodopi, 2007.151-74. Print.

--. "The Computer-based Analysis of Narrative and Multimodality." New Perspectives on Narrative and Multimodality. Ed. Ruth Page. London and New York: Routledge, 2010. 50-64. Print.

Snyder, Joel, ed. "Audio Description Guidelines and Best Practices." Web. 10 Aug. 2013.

Vandaele, Jeroen. "What meets the eye. Cognitive narratology for audio description." Perspectives: Studies in Translatology 20.12012: 87-102. Print.

Vercauteren, Gert. "Towards a European guideline for audio description." Media for All: Subtitling for the Deaf, Audio Description and Sign Language. Eds. Jorge Diaz Cintas, Pilar Orero, and Aline Remael. Amsterdam: Rodopi, 2007. 139-50. Print.
Table 1: Frequencies of troponyms of "looks" in descriptions A and B.

Troponym      A     B

stares       16     4
gazes         9     5
peers         8     3
glances       5     2
eyes          4     0
studies       3     0
glares        2     0
regards       0     1

Table 2a: Frequencies of word forms ending -ly that follow the
word forms "looks."

directly                                                             9

anxiously                                                            8

sadly, steadily, thoughtfully                                        7

nervously                                                            6

fearfully, grimly, longingly, quizzically                            4

curiously, slowly                                                    3

blankly, briefly, carefully, frantically, intently, pensively,       2
searchingly, seriously, sheepishly, sympathetically, uneasily,

accusingly, appealingly, awkwardly, bitterly, calmly, cautiously,    1
closely coldly, contemptuously, defiantly, disappointedly,
disapprovingly, earnestly, enquiringly, entreatingly, firmly,
forlornly, gravely, guiltily, impatiently, incredulously,
malevolently, meaningfully, miserably, mournfully, pathetically,
pityingly, pleadingly, pointedly, questioningly, quickly,
resignedly, skeptically, shakily, sharply, shyly, solemnly,
surreptitiously, tenderly, unsurely, vacantly, warily, wistfully

Table 2b: Frequencies of word forms ending -ly that follow the
word form "walks."

slowly                                                              57

purposefully                                                         9

briskly                                                              8

carefully, cautiously, confidently, quickly, stiffly                 4

hesitantly, nervously, steadily, tentatively, unsteadily             3

awkwardly, casually, determinedly, excitedly, gently, hurriedly,     2
sadly, uncertainly, warily

apprehensively, calmly, despondently, dramatically, dreamily,        1
drunkenly furtively, gingerly, gleefully, happily, languidly,
limply, nonchalantly, pensively, proudly, quietly, resolutely,
shyly, smartly, softly, solemnly, suspiciously, swiftly,
thoughtfully, unhurriedly, wearily

Table 3: Frequencies of word forms ending
-ed that follow the word form "looks."

puzzled                                                  17
confused                                                 15
troubled                                                 11
shocked                                                  10
concerned                                                 7
embarrassed, surprised                                    6
amazed, disappointed, worried                             5
bemused, pleased, stunned                                 4
annoyed, dazed, perplexed, terrified, unconvinced         3
agitated, alarmed, exasperated, horrified, interested,    2
  relieved, unimpressed, unnerved
aggrieved, amused, ashamed, astonished, baffled,          1
  bewildered, bored, captivated, chuffed, defeated,
  deflated, delighted, determined, disgruntled,
  disheartened, dismayed, dispirited, distracted,
  disturbed, drained, entranced, excited, flattered,
  frightened, impressed, irritated, moved, mystified,
  peeved, perturbed, petrified, saddened, scared,
  shell-shocked, startled, strained, stumped,
COPYRIGHT 2015 Northern Illinois University
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2015 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Palmer, Alan; Salway, Andrew
Article Type:Essay
Geographic Code:1USA
Date:Jun 22, 2015
Previous Article:Hypallage and the literalization of metaphors in a Dickens text.
Next Article:Allusions and meaning in Conrad's The Shadow-Line.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters