Tourism English and IT evaluation.

Abstract

An oral fluency evaluation for Tourism English (TE) is proposed by contrasting linguistic production with corpus data. Electronic written and oral text collections (corpora) are designed and exploited so that an evaluative approach may be devised and carried out. The aim is to identify any significant oral performance changes arising between learners who have managed such electronic resources and learners who have not.

Introduction

In tertiary education, two main branches of ESP (English for Specific Purposes) seem to be converging. These are EAP (English for Academic Purposes) and EPP (English for Professional Purposes). As Dudley-Evans and St. John explain (1998, 174), one of the reasons for this convergence is the rapid change taking place in university studies, especially technical ones, where "carrier content dates rapidly'. Nelson (2003) particularly refers to the case of Business English, a type of specialized communication in itself that brings a "special jargon" to the class. According to Nelson, the use of a special type of discourse is also perceived as a significant feature of oral business communication. In the English for Tourism classroom, the situation of oral skills demanding specialized knowledge is also evident. As examined in previous work (Curado 2003a; Curado 2003b), such a content specialization may range from giving a businesslike report to speaking about different promotional topics (e.g., planning excursions, monument description, etc).

Focusing on these oral skills leads to having to adopt strict evaluative guidelines that assess student knowledge. In my teaching context, the use of specific corpora (homogeneous text collections) can work as significant reference for oral language evaluation. In particular, I have focused on two types of specialized corpora: Spoken business reports and product reviews found on the internet, and academic discussions about socio-economic topics taken from MICASE (2002). This method proposed takes word frequency as a key reference for language command assessment (Bley-Vroman 2002). My findings in this paper are discussed in the light of feasible language acquisition.

Corpora used in the evaluation

My TE (Tourism English) course is chiefly based on both business and promotional language. The textual material would thus be restricted in terms of text type choice and specialized language use: Texts on Business technology for the Tourism sector, and a student discussion group on Economics issues. Building such a restricted corpus is crucial, as Thompson (2002, 15) and Hunston (2002, 4) assert, in the ESP learning setting. For the evaluation of oral performance in TE, the two corpora mentioned should be built from already existing material, consisting of transcripts of recorded speeches. Some web resources (e.g., Global Edge 2002) are favored by learners in the area of technology, especially in its relationship with the workplace. This first oral corpus is made up of 25,265 words, taking, overall, two hours and 15 minutes to deliver. These texts do not, however, entirely constitute spontaneous speech, since some documents are read out loud like a conference paper. The second corpus does feature spontaneous speech in the form of study group discussions on socio-economic issues, and classroom discussion units about Economics. This corpus contains 22,526 words, and is two hours and five minutes long.

Evaluation procedure

Two oral tasks are assigned to spur effective communication for the business setting in Tourism-related jobs; thus, the evaluation focuses on content and language production at the business place. The first task is an oral report (monologue), where I seek to assess knowledge of typical corpus language. Three main levels of word use are considered: 1., Lexical constructions that are highly frequent in business-related corpora, but not specific of a single domain or subject in business (semi-technical); 2., phrases or clusters that are highly frequent in different types of texts without being ascribed to any particular subject area (academic); and 3., word combinations that are highly frequent in one subject or domain alone (technical). Content words (i.e., nouns, verbs, adjectives, adverbs) are the primary material in this task, as those items tend to embody key stored information about the words of a language (Garman 1990: 137). The second oral task requires learners to conduct a five- to ten-minute debate or discussion in groups, where a crucial aim is to evaluate conversational fluency. In this case, frequent function items employed by native and non-native speakers of English at a given instance of academic discourse exchanges (such as those included in the MICASE corpus) are accounted for. My TE students are divided into two groups of ten subjects each: Control and experimental group. These learners belong to TE III, an upper/intermediate level course taught at my institution. Both groups are assigned the same two oral tasks: Giving an oral presentation (a five- to ten-minute lecture on a Business technology topic), for which they have two weeks to prepare, and a second oral task for which they have to get together in groups of five and carry on a spontaneous conversation about an academic topic related to their studies (e.g., rural tourism, eco-tourism, etc).

The experimental group, in addition to having to do these two oral tasks, undergoes a previous two-week period of explicit corpus-driven activities. According to Campbell & Stanley (1966, 39-45), a contrast of this type of group-based work, should provide feedback as regards effective linguistic production. For example, frequent lexical constructions, considered semi-technical (e.g., run the company), academic (e.g., in this paper we present), or technical (e.g., technology-marketing profile) become significant in terms of statistical frequency. The key objective is for students to be able to realize patterns (either repetitions or synonyms) in short (oral and written) texts. The main tool employed by the experimental group is WordSmith (Scott 2000), providing a resourceful identification of keywords and clusters. In this case, texts and paragraphs from the corpora are selected so that specialized collocations as well as non-specialized clusters be identified (i.e., semi-technical, technical, and conversational items). Semantic aspects of collocations can also be approached by encouraging learners to deduce verbal information from commonly seen patterns in the corpora. Hints at common semantic aspects associated with these structures tend to assist learners in the visualization of patterns. In addition, done with four students only, aural / oral activities seek to examine if these four learners can improve their pronunciation in the first oral task in comparison with the other six who did not work on oral / aural activities. An example is the revision of third person singulars and plurals in words like purchase, manage, resource, service, important because those word endings are often mispronounced or omitted during the talks.

Results

The first oral task includes 20 individual reports about Business technology, spoken from notes. The second task leads to four five-member group discussions about Tourism-related topics like the promotion of Rural Tourism, types of Tourism that generate more capital, things that hotels should improve, and things to improve in one's hometown regarding historical landmarks. These talks were produced spontaneously and without notes. Two approaches to the evaluation are devised, depending on the type of task performed: In the first one of reports, three different aspects were examined: First, scores based on the total of effective (corpus-based language), good (correct but non-corpus-based), and weak constructions used by the Control and Experimental groups. Secondly, points derived from the amount of semi-technical and technical word combinations (collocations and noun compounds) generated, as well as from the phrases or clusters used. Thirdly, the number of pronunciation deviations and pauses made during the lectures. In contrast, for the second task (group discussions or debates), only one lexical category was taken as reference for evaluation: Cohesion-marking words. Considered 'effective', the corpus-based language used by the experimental group amounts to 37 constructions (e.g., provide + information, run + business / application / program, new information technologies, sales trends, available on the Internet), whereas only 18 were uttered by the control group. In terms of 'good' or "correct" language, more examples (348 vs. 342) were checked in the experimental group. Regarding mistakes, however, this group had more (73 vs. 62). Only three collocations were stressed or pronounced incorrectly by the experimental group: available on the internet, dealt with the, scan the images. Interestingly enough, it should be pointed out that the four students who practiced the aural / oral drills did not make any mistakes in the pronunciation of corpus-based constructions.

For the assessment of the second task, some learners' function words were registered as significant marking devices throughout the conversations. Such elements were also divided into effective, good, and weak. In the first category, the experimental students' scores outnumbered the control group's: 15 effective markers vs. three. Good markers, however (those not found in the oral corpus of conversations), were a bit less frequent (11 vs. 14); finally, the control group students made more mistakes in marking speech by means of corpus-based elements (13 vs. 22). An overall positive achievement is the experimental students' greater use of effective lexical elements and linking devices. An example is the second person pronoun exploitation to explain things: like you have for instance, if you think, what you're talking about, you know, etc. Other effective language (not used by the control group) was phrased as so that's the point, I don't think so, Okay so, what I mean, kind of like, kind of, things like that, but like. In turn, more mistakes were made by the control group, especially due to lack of grammaticality in the utterances (e.g., no matter that*, me think that*, how do you this*, from my idea*, etc). The control group also paused more during the conversations, mainly due to their not being able to find a new idea or to put ideas together (33 pauses vs. 18).

Discussion

In general, I deduce from the findings above that corpus-based information, if conveniently exploited in class, provides a positive output for oral language development. This is above all revealed by the larger number of effective corpus-based constructions achieved in the presentations. Indeed, when asked about especially helpful items in their presentations, the experimental students made explicit reference to some collocations exploited, such as run the company, manage the data, in great detail, and so forth. In contrast, their higher rate of mistakes computed in the first task, may be due to their generally faster pace. In fact, a richer lexical intake was demonstrated by the experimental group, and, as a result, the larger number of pauses made by them in the first task is perceived as a consequent factor of their searching for effective lexical output, as some authors claim (e.g., Garman 1990). This tends to happen in the production of oral discourse when new lexical information is stored. An example is the search for the expression cope with the issue, which a student could not produce even when she had it at the tip of her tongue. Instead, she uttered form with the issue*, a mistake probably caused by nervousness, but also by in-depth lexical searching.

The results also demonstrated that the experimental students were more capable of specialist and non-specialized knowledge production by means of collocations and clusters, whereas only three subjects from the control group showed this competence. Students, however, do not seem to notice the importance of clusters as evidently as that of semi-technical collocations (as deduced from comments in class). The appropriate construction of noun compounds by the experimental group is also related to corpus familiarization, in contrast to the other students' lack of technical language in the talks. In addition, the fact that the four students who worked on pronunciation drills did not make any oral mistakes in the key constructions, suggests that more emphasis should be laid on aural and oral activities during the corpus-driven exercising period. In the second task the experimental learners also had a high performance, mainly realized by the textual cohesion measured through their use of marking devices. Some students actually commented on their grown confidence in the keeping up with dialogues, making explicit reference to the aid of linking devices such as you know and like you know to continue an explanation or starting a new point. In contrast, the control group paused more in the connection of discourse. Together with fewer cohesive function items, they showed a lower degree of confidence in spontaneous speech.

Conclusions

This paper has provided a summary on how to use two specific corpora for the evaluation of oral language performances in Tourism English. Such an approach has been taken by mainly relying on word frequency as an evaluative yardstick to check if students can produce corpus-driven data, previously explored in class. Ten learners were exposed to a two-week period of in-class corpus activities (Experimental group), whereas 10 other students were not (Control group).

As a first main reflection, 1 have noticed that the overall results are positive for ESP teaching and learning. Second, because the experimental group produced a greater deal of semi-technical collocations, clusters, and technical compounds in the first oral task, I assume that the two-week trial period of in-class corpus-based development yielded beneficial intake for oral tasks. Third, although the two groups made a similar number of language mistakes, the former in the oral reports, and the latter in the discussions, 1 perceive a qualitative distinction between the two cases, mainly, in that deviations in the experimental students are caused by: I., their overall faster pace in the presentations, 2., the longer duration of the speeches, and 3., more pauses made in order to find effective lexical output (indeed, as some authors observe in the production of oral discourse, this feature is common among speakers when a large amount of lexical information has been recently stored). Fourth, in the first task those students without a previous pronunciation practice of corpus-based language made more mistakes, leading to the assumption that, during the two-week corpus exploitation period, it was important to integrate oral / aural activities with the written exercises. Fifth, the experimental group showed more confidence than the control group in the second task, corroborating in questionnaires that some examples of the linking devices examined in the corpus actually became useful for the connection of ideas and thought.

As a result, I believe that future work should focus further on corpus and technology use for oral language improvement. Investigation of oral communication in ESP settings should include a double-fold approach: To analyze technical language as it is produced in a corpus of prepared speech (e.g., lectures), but also in spontaneous conversations (e.g., meetings). In the case of Tourism, the first situation may include business management exchanges, whereas the second would have a much less formal tenor (e.g., conversations with tourists, guided tours, etc). By exploring discourse and evaluating it in class, I may thus pinpoint the academic and professional communication needs that my courses should demand.

References

Bley-Vroman, Robert. "Frequency in Production, Comprehension and Acquisition". Studies in Second Language Acquisition, Vol. 24, No 2 (2002).

Campbell, Donald T. and Julian C. Stanley, J.C. Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally, 1966.

Curado, Alejandro. "The Use of Corpora and IT in a Comparative Evaluation Approach for Business English Oral Reports". RECALL Journal, Vol. 15, No 2 (2003a).

Curado, Alejandro. "Academic Language on the Web and Task Competencies in ESP". In Jordi Piqur-Angolans, M. Jose Esteve, and M. Luisa Gea-Valors, Eds. Internet in Language for Specific Purposes and Foreign Language Teaching, 15-26. Castello:Coleccio Estudis Filologics, 2003b.

Dudley-Evans, Tony and Mary J. St. John. Developments in ESP. A Multidisciplinary Approach. Cambridge: Cambridge University Press, 1998.

Garman, Michael. Psycholinguistics. Cambridge: Cambridge University Press, 1990.

Global Edge. Global Edge resources for Economics. Accessed on 14 May, 2002. http://globaledge.msu.edu/ibrd/ibrd.asp

Hunston, Susan. Corpora in Applied Linguistics. Cambridge: Cambridge University Press, 2002.

MICASE. Michigan Corpus of Academic Spoken English. Accessed on 20 September, 2002, http://www.hti.umich.edu/m/micase/browse.html.

Nelson, Mike. "Worldly Experience". Guardian Weekly [Online]. Accessed on 20 March, 2003. http://education.guardian.co.uk/tefl/story/0,5500,917515,00.html

Scott, Mike. WordSmith Tools 3.0. Oxford: Oxford University Pros, 2000.

Thompson, Paul. "What Use are Corpora in the Teaching of EAP . In Andy Cresswell, Silvia Bernardini, and Guy Aston, Eds. Proceedings of the Fifth Teaching and Language Corpora Conference, 71-72. Bertinoro, Italy: University of Bologna, 2002.

Alejandro Curado Fuentes, University of Extremadura, Spain

Alejandro Curado Fuentes is Associate Professor of English for Specific Purposes.